LUVLE Updates - Michaelmas 2011 and Lent 2012 | Learning Technology Group | CIS

LTG have made significant improvements to the LUVLE server environment, our code base, and our notification systems over the Summer. This update is intended to help you understand what we are doing to: first, notify you of any problems encountered, second, minimize the impact of any outages on you, our users, and third, find the root of any problems that appear. It has been written in a blog-like format with the most recent updates at the top and the older ones below that. Each update will start with the date.

For previous updates see the Summer 2011 LUVLE Update , Michaelmas 2010 LUVLE Update or Lent 2011 LUVLE Update pages.

Wednesday, 25 April

The myplace.lancs serverr which hosts MyPlace, MyPGR and Appraisals will be unavailable from 15:00 to 17:30 today to receive important Appraisals updates. We are sorry for the inconvenience caused.

Wednesday, 18 April

The LUVLE, MyPlace, MyPGR and other servers received their monthly Window updates today starting at 16:00 with all servrs back in service at 17:04.

Friday, 30 March

The myplace.lancs server was re-started for unscheduled maintenance today . We noticed some irregularities in performance and as a precaution re-started the server to clear the memory. It was down for about 11 minutes starting at 12:03.

Friday, 23 March

The luvle.lancs server, which services primarily LUMS and FST, had a crash at 11:03 lasting 8 minutes and affecting up to 327 users. The source of the problem is unknown and the server logs have been forwarded to IBM for further analysis. It was automatically restarted.

Friday, 2 March

The domino.lancs server, which supports LUVLE Spaces primarily with FASS and SHM, had a crash at 16:33 lasting approximately 57 minutes. It appears the problem was initiated by a network change.

Thursday, 1 March

The luvle.lancs server, which supports LUVLE Spaces primarily with LUMS and FST, had a crash at 12:57 lasting about 7 minutes and affecting up to 255 users. It was automatically restarted. The crash file has been sent to IBM for analysis but it appears as if it was associated with an LDAP request.

Wednesday, 15 February

Our MyPGR update took about 40 minutes, a little longer than expected. However it went well and now there are some new pop-up reminders and an easier interface to the list of names. In fact we've already had some compliments!

Wednesday, 15 February

One LUVLE server, domino.lancs, which services FASS and SHM, had an unexpected outage today at 12:46 lasting 10 minutes and affecting up to 179 users. It was automatically re-started at 12:56. Our initial investigation indicates that a particular file may have caused the outage but we have requested analysis by IBM. We will advise if this is the cause of the problem and our plan to deal with it on all LUVLE servers.

MyPGR will be unavailable at 15:00 on 15 February for about 30 minutes to update some code. We will be adding the much heralded ad hoc Appraisal and most of the reports featured in our training session last week.
We regret any inconvenience that might result from this important upgrade.

Monday, 13 February

At 09:49 luvle.lancs experienced an outage affecting up to 148 users. All service was restored in 11 minutes. Our initial look into the server logs show nothing obvious as the cause, but we are now comparing it to the last similar occurrence in 27 Sep 2011 to see if there are any patterns. We will report back if there are any insights gained.

Wednesday, 25 January

The Windows updates on the five LUVLE servers were all successfully completed in 43 minutes. Now we're good for another month!

Monday, 23 January

ISS are updating the Windows server software on all LTG services starting 15:00 on Wednesday, 25 Jan. All LTG services, including LUVLE, MyModules, LUVLE Spaces, Computer-Based Assessment, LUMES, MyPlace, and MyPGR will be affected. We expect the disruption to individual servers will be about 10-15 minutes, but due to the number of servers affected the total task may extend over a period of up to 1.5 hours in the worst case.
ISS regrets any inconvenience that may be caused by this periodic maintenance.

Wednesday, 18 Januaruy

At 15:00 the myplace.lancs server was taken down for scheduled maintenance so we could install the latest version of MyPGR Appraisals. It was completed in 96 minutes.

Wednesday, 11 January

At 9:47 am the ldomino.lancs server had an outage affecting up to 25 users - another http problem. This time the newly installed automatic restart routing had operations restored in only 8 minutes!

Tuesday, 20 December

At 23:19 the domino.lancs server had an outage at 10:20 lasting 43 minutes and affecting up to 18 users. It was another http problem, meaning that the server software was running well but the connection to the internet was out. This has the effect of keeping new users from logging on. We have been working over the last week to develop a new tool to automatically restart servers with http problems, but had not finished all the testing by the time this crash occurred.

At 11:38 we had installed the new automatic restart routine on all five servers!

Saturday, 17 December

At 23:19 the luvle.lancs server had an outage lasting 4 hours 55 minutes. The outage affected the web portion (http) of the server and meant that no web access was available for users of this server (LUMS and FHM). There were approximately 35 users logged on at the time of the outage. This was a very unusual outage because Domino did not recognise there was a problem and continued as normal throughout. There was no indication of a problem in the Domino server logs, also very unusual, and so our current hypothesis is that an event outside of Domino caused the problem. There were no calls or emails to the Service Desk about a problem during this time. Normal service was resumed at 04:16 following the usual scheduled nightly server restart. We are investigating further and will report any further findings, if any surface.

Monday, 5 December

We experienced something unusual today starting at 10:43 when about 298 people were logged on. It looked like a server outage even though the server never stopped. Thanks to the seven students who called the Service Desk, we became aware of a problem in which these students were being denied access to LUVLE on one of the servers. They were getting a message that their username or password are incorrect. While a re-start of the server at 11:29 seemed to clear up the problem, we noticed there were about 1,800 emails being generated within LUVLE. Upon further investigation we found that we experienced a possible Denial of Service attack through one of the forms on the LUVLE login page. We took that page down and expect no further problems. The attack seems to have originated from an external Amazon-hosted source and NOT from on-campus. We have submitted an abuse report to Amazon and are awaiting their reply.

Wednesday, 23 November

LTG took the opportunity of moving the CETAD LUVLE Spaces (see below) to also update the Windows software on the KTRU server. While that significantly extended the maintenance period, it eliminated the need to take the server down at a later date. All CETAD LUVLE Spaces - 33 databases and 730 Mb - seem to be operating well. Although they have been moved to a different server, users will NOT have to worry about the changed address since we are automatically re-directing all browser request from the old to the new address.

Monday, 21 November

LTG will be conducting maintenance on two LUVLE servers from 15:00 on Wednesday, 23 November. The purpose of this action is to move the CETAD LUVLE Spaces to a more robust server. After the change, users will continue to use the same web links as before (we'll put re-direct instructions in the system). We expect the entire task to take about 15 minutes.

Tuesday, 15 November

At 10:58 today the ktru-main.lancs server, which services CETAD and other modules with off-campus registrations, had an unexpected outage lasting 15minutes and 2 seconds. THere weere 8 users logged on at the time. The source of the problem appears to be similar to the memory leak we experienced last year, perhaps as more CETAD modules are using the full capabilities of the LUVLE 11 template, including X-Pages. We have placed this server on the 4am restart schedule as a precaution.

Wednesday, 9 November

At 12:08 today the mymodules.lancs server experienced a 1 minute 11 second outage. The problem appears to have been due to an OLE call to another database which hung up. While there were 432 users logged on at the time, it is unlikely many were actually affected for two reasons. First, it was re-strted in just over a minute. Second, most users log in to MyModules for the purpose of immediately accessing their LUVLE Spaces on another server, although the logon is carried for 30 minutes.

Thursday, 20 October

At 08:36 today the LUVLE server hosting MyModules had an unexpected outage. It was automatically re-started and lasted just over one minute. Preliminary investigation indicates it may have been caused by a student timetable request, but we have sent the files to IBM for analysis. We will keep you advised if IBM turns up anything of note. There were about 77 users logged on and it is likely that few would have noticed because of the short duration of the outage.

Thursday, 6 October

Today at 14:00 for about 15 minutes we will have to temporarily take down the mymodules.lancs server for some last minute maintenance in order to correct a problem which could affect timetable accessibility. We are making this change in order to better serve students who historically access their MyModules over this weekend (just before term start).

We also remind you that timetable info is now available on most mobile devices; for more information and download instructions see http://www.lancs.ac.uk/iss/services/mobile/ilancaster/

Wednesday, 5 October

Today between 16:00 and 18:00 we will be doing some Windows updates to all LUVLE servers. While we regret any inconvenience, this will allow us to start the new academic year with an updated set of server software.