Dear Students and Staff
First, we want to apologise for any disruption or extra work which problems with LUVLE service may have caused you. Please be assured we are doing everything possible to, first, minimize the impact on you, our users, and second, find the root of the problem. This update is intended to help you understand what we are doing to reach those two concurrent goals. It has been written in a blog-like format with the most recent updates at the top and the older ones below that. Each update will start with the date.
For previous updates see the Michaelmas LUVLE Update or Lent LUVLE Update pages
Tuesday, 27 Sept 2011
Today at approximately 12:11 luvle.lancs lost its connection to the internet (called the http connection) although the server did not go down. Our performance monitoring software notified us and we re-started the server manually. It was returned to full service within about 4 minutes. We have sent a request to IBM to help us identify this problem. We have also sent a request to the IBM senior executive for HE to determine if there is a pattern that can be discerned from these five recent outages. We'll keep you posted.
So far Friday's server fix seems to be holding.
However Thursday's outage is still perplexing the IBM staff, and they have progressed it from L3 support to the development team who actually write the software.
Monday, 26 Sept 2011
Friday's outage appears to have been caused by a corrupted database file. The LTG staff noticed the similarity between this outage and one on 1 Sept, compared processes, identified the corrupted file that caused the server hang-up, and replaced it. We are watching that server (and process) carefully, but feel encouraged that we have identified it as time approaches the beginning of term.
The outage on Thursday has been referred to the highest level of Service Desk programmers (L3)within IBM, having stumped the L1 and L2 levels. We continue to stay in communication with them.
Friday, 23 Sept 2011
At 11:52 today the luvle.lancs server experienced an uncontrolled outage for approximately 21 minutes affecting up to 24 users mostly from LUMS and FST. The server has been manually re-started and is functioning fine now. The cause of the outage is not evident, although we are doing further examination, and the appropriate logs have been sent to IBM for analysis as a L2 priority request. We will keep you posted of all findings.
Thursday, 22 Sept 2011
Today at 11:40 the luvle.lancs server experienced an uncontrolled outage for 4 minutes affecting up to 22 users, mostly from LUMS and FST. Since it lasted for only 4 minutes, it is likely that few knew there was a problem. The cause of the outage is not evident, and the appropriate logs have been sent to IBM for analysis. We will keep you posted of their findings.
Friday, 09 Sept 2011
This is a reminder that there will be a campus network failover exercise tomorrow, Saturday 10 Sept. All LUVLE users are advised to avoid using LUVLE between 09:00 and 17:00. See the 1 Sept advice below for details.
Also be aware that LUVLE, MyPlace, MyPGR, LUMES and Questionmark Perception will be unavailable on Wednesday, 14 September from 8:00 am for up to one hour. We will be performing system updates and reconfiguring hardware at that time.
Wednesday, 7 Sep 2011
Today at 13:48 we had a short 3 minute outage affecting up to 10 users which was caused by a corrupted file on the domino.lancs server. The good news is that our server software noticed the problem and initiated a re-start so quickly the most users probably did not notice the problem. We sent the files to IBM and they identified the problem as the corrupted file, which we have fixed. We believe this is a one-off that should not recur. Naturally, we will be on the lookout, though, especially as we approach the new term.
Thursday, 1 September 2011
Today at 16:55 we experienced our first unmanaged outage since May. It lasted 21 minutes affecting up to 43 users. We had noticed that the HTTP (internet connection to that server) was non-responsive and after 5 minutes began a
restart which hung up the server further. It appears to be an OLE problem but we need further analysis from IBM.
Thursday, 1 Sept 2011
ISS is undertaking a failover exercise of its virtual infrastructure again this year. Since all LUVLE servers were virtualised since the last exercise, they will be included in this important test designed to give you increased reliability. It will take place on Saturday September 10th from 9am to 5pm. Any of the users of LUVLE must be aware that LUVLE may not be available during this period and any use started during this period may not be saved. Thus we are suggesting all users avoid using LUVLE between 9 am and 5 pm.
The failover exercise will involve failing over all virtual machines (VMs) to one datacentre, verifying the failover, and then returning the VMs back to their original location. Failing over a VM will involve both shutting down the machine and a reboot. The exercise is then repeated to the other datacentre.
Thus LUVLE will be AT RISK all day, but down for about two hours. However, we are not sure of start time for the LUVLE servers.
Friday, 12 August 2011
LUVLE will be unavailable on Wednesday 17 August from 3:00 pm for up to an hour. We will be installing some important software updates and increasing the resources for the MyModules server.
Wednesday, 10 August 2011
Announcement to Staff: Create your 2011-12 LUVLE Spaces Now!
Departmental Administrative or Teaching Staff now have full control over when they create their 2011-12 LUVLE Spaces by going to their MyModules page or clicking on this LUVLE Space Creation link. This is now a short and simple task. A two-minute movie is available to describe and demonstrate what you need to do (http://luvle.lancs.ac.uk/luvlespaces.nsf).
Content can be transferred from your 2010-11 LUVLE Spaces any time after you have created your 2011-12 LUVLE Spaces by clicking on the Transfer Content link in the Tools section of the newly-created LUVLE Space. Staff have full control over what is transferred and hidden. A two-minute movie is available to describe and demonstrate what you need to do (http://luvle.lancs.ac.uk/luvlespaces.nsf).
Archiving of the 2010-11 LUVLE Spaces will take place on 16 September when any 2011-12 module LUVLE Spaces you have created are populated with the Michaelmas student registration data from LUSI. You will NOT need to take any action at this time; archiving will happen automatically.
If you create your LUVLE Space after 16 September, registered staff will have immediate access but the LUVLE Space will not be populated with the student registration data until the following morning (thus students won’t be able to see it in their MyModules or access it until a day later).
BACKGROUND
Our Learning Technology Advisory Group had three suggestions in May on how to improve the LUVLE service for 2011-12. Suggestions included the following:
· Develop a timetable for LUVLE over the Summer;
· Communicate the timetable, and
· Make LUVLE Spaces available earlier, separating LUVLE Space creation from the LUSI Rollover (typically 15 September when the 2010-11 student data is rolled over to an archive and the 2011-12 student data are made available in LUSI) to give staff more control over the creation and content transfer processes.
This announcement fulfils those three requests from our users.
DETAILED INSTRUCTIONS (available only in this web link)
Creating your LUVLE Spaces - LTG have developed a tool that allows the 2011-12 module LUVLE Spaces to be created earlier. We have been trialling this tool with staff who have already created their new LUVLE Spaces.
In the past, LTG aggregated together all departmental requests for LUVLE spaces for the coming academic year and then all sites were created in a single two week block by LTG. This often meant that requests were inaccurate and omissions were made. This in turn meant that sites were created that were not needed or had to be created during the year, often at the last minute. Another problem was that some staff needed their sites created earlier than that date, due to earlier start dates or staff being unavailable due to other commitments. This tool simplifies the process by empowering Departmental Administrators or Course Convenors to set up their modules for the coming year at their convenience. This tool can be accessed from your staff MyModules pages or by clicking on this LUVLE Space Creation link.
From: ISS Service Desk [mailto:iss-service-desk@lancaster.ac.uk ]
Sent: 09 August 2011 10:12
To: Jones, Tom; Bassey, Shirley
Subject: New LUVLE Space for MUSC126 The right to Welsh music as a human right
A new LUVLE Space has been created for: MUSC126 and staff should now be able to login to this site.
Students will be able to login soon after the 'LUSI roll-over' to the new academic year around the 16th of September; 2011-12 spaces will appear on MyModules pages and on the LUVLE Home catalogue page at about the same time.
Until MyModules and the LUVLE Home catalogue page show the new spaces, you can use the LUVLE Spaces Creation application to check on the status of all of your department's LUVLE spaces for 2011-12; you can also use this application to create new LUVLE spaces.
If you have any problems with setting up your new module spaces, please contact the ISS Service Desk.
from the Learning Technology Group, ISS
If you have any problems with setting up your new module spaces, please contact the ISS Service Desk.
Transferring your content – Once you have received the email saying your LUVLE Space has been created, you can transfer content from those 2010-11 LUVLE Spaces that you are registered as teaching on in LUSI. The Transfer Content tool is located in the Tools section of the new LUVLE Space. You will be able to transfer Announcements, Course Materials and Links (if they were enabled in your 2010-11 LUVLE Spaces). This facility is only available to staff with administrative privileges on the new LUVLE Space. As before, you still have the option to display or hide the migrated materials as best suits your instructional needs. More information is available in the new LUVLE Spaces.
If you have content in your 2010-11 LUVLE Spaces that you have migrated to your new LUVLE Space but you do NOT wish them to be viewed when archived, you may Hide or Delete them at this time (however, we suggest the Hide option in case you have a problem and need to go back to them).
Archiving your old (2010-11) LUVLE Spaces –Archiving your old LUVLE Spaces will take place after 16 September, when student access to your old LUVLE Spaces will automatically be changed to ‘Read Only’. You will have to do nothing further to accommodate this archiving. Archived 2010-11 LUVLE Spaces will be available to staff and students on 16 September when the new (2011-12) LUVLE Spaces are available in MyModules (IF you have created them!).
End of Announcement..............................................................................................................................
Wednesday, 27 July 2011
The Windows updates of our servers went well and the task was complete within the scheduled hour.
Monday, 25 July 2011
We have to do some monthly Windows updates on the LUVLE servers, so we have scheduled it for the normal maintenance period. Access to all LUVLE services may be unavailable on Wednesday, 27 July from 15:00 for up to an hour. We apologise for any inconvenience.
Friday, 24 June 2011
We had a problem with one of the servers, called KTRU-Main, in the wee hours of the morning. This server hosts fewer than 5% of the LUVLE Spaces, and is focused on supporting distance learners. More importantly, most of you will not have been affected.
Around 23:45 on Friday night, KTRU-main lost its connection to the server which does the campus authentication. As a result, anyone who was trying to sign-on directly to a LUVLE Space on that server using a direct link would have gotten a 'password incorrect' error message. However, the server continued functioning and those who logged on through MyModules, and our data indicates that is in excess of 80% of you, would have logged in successfully. Applying some Windows patches and rebooting at 07:45 on Saturday morning solved this problem. LTG are discussing this problem with the people who run the authentication service, but we feel confident few if any of you were actually affected. On the other hand, if you were affected and if you got some message other than a 'password incorrect' error message, please let us know.
Wednesday, 22 June 2011
We are pleased to announce that the move of MyModules to a new server on its own was successful. we will continue to watch th performance of this new server to be ssured it is working well.
Tuesday, 21 June 2011
NOTICE: 3rd year students will lose their access to LUVLE & My Modules on the 30th June when the summer term ends. Those 3rd year students wishing to download lecture notes, coursework submissions, etc from LUVLE for future reference need to do so before 30 June.
Also note, exams marks will be made available over the Summer on the Student Registry website, NOT LUVLE. If you have a question relating to transcripts and marks please contact your department or the Student Registry.
Tuesday, 21 June 2011
We are pleased to announce that we are moving MyModules to its own enhanced virtual server tomorrow, Wednesday, 22 June between 15:00 and 16:00. What that means to you is two things: 1) it is a faster and more capable server which should be able to serve up your timetables etc quicker, and 2) if there is a problem with any of the servers providing LUVLE Spaces, MyModules will still be able to provide access to LUVLE Spaces on other servers. If you have any problem with MyModules shortly after that timeframe, please be sure to report it immediately to the ISS Service Desk (and if you get an error message, please tell us exactly what it says). Thanks!
Friday, 20 May 2011
At approximately 15:00 today, one of the LUVLE servers (luvle.lancs) had a controlled outage in which our performance monitoring software triggered a re-start. It lasted almost four minutes and 294 users were affected. It appears to have been caused by an indexing issue, and we continue to monitor for it.
Monday, 9 May 2011
We have just suffered our first outage of the term after having none from 22 March. It happened in the wee hours of the morning at about 04:10 so few if any of you actually knew. This was an extended scheduled outage affecting only one server. The normal daily restart wasn't a clean one; we are investigating the cause.
Monday, 25 April 2011
We are pleased to announce that virtualisation has gone well with only some minor glitches. In addition to putting all LUVLE services on newer hardware, it will allow us to adjust resources more quickly. We have also gathered some important load test data which we are analyzing to determine the optimum normal configuration, especially in relation to the amount of RAM and the number of CPUs assigned to each of our LUVLE 10 servers. We expect the full analysis to be completed and reviewed within the next week.
In the meantime, we wonder if you noticed that there is a new LUSI web service that displays not only student and staff timetables, but also rooms, modules and programmes timetables. Try https://timetabling.lancs.ac.uk/att1011/ We understand this will soon be available on your mobile smartphones. More info to come when it becomes available.
Wednesday, 13 April 2011 - PM
We had a small problem with one of the newly virtualised servers late this afternoon. To fix it we restarted the HTTP task at about 15:51 today. That means that some users may have received error messages for a few minutes before the restart which we initiated. A student user called the Service Desk and identified the type of message he saw, allowing us to react immediately. (Thanks!) We replaced a part of the code in the template and it seems to have fixed it for now. It was probably a result of our recent virtualisation in which we moved files from two servers onto one. We are watching the server more closely through the evening now. Neither of the other two servers is displaying this problem, but we'll keep an eye on it.
Tuesday, 12 April 2011 - PM
We are pleased to report that the virtualisation of domino.lancs has gone well. We are using all virtualised servers now. See my 6 April expalanation as to why we are pleased. Now we have to analyse the test data we gathered to determine how we are going to use it to improve the responsiveness of the LUVLE servers.
Monday, 11 April 2011 - PM
We are pleased to report that the virtualisation of the two domino servers into one has gone extremely well. In fact, some of you are using it right now (even though we said it was going to be unavailable today and at risk tomorrow!).
Tomorrow we will move on to a series of tests on the new domino.lancs server. While testing was always in the plan, the fact that we were able to get this virtualisation done so early today allows us to run some additional important stress test tomorrow instead of later, keeping us from making announcements and shutting down the server again.
At 09:30 Tuesday morning we will move a copy of MyModules to the other newly virtualised server (luvle.lancs) to allow those on luvle.lancs to access their module spaces while we are testing on domino.lancs. Here's which LUVLE Spaces WILL be available on Tuesday: LEC, Psychology, Physics, Marketing, Management Science, OWT, History, Accounting & Finance, Economics, Entrepreneurship, Management and Organisation, CEEC, and Health Research.
But remember that if your LUVLE Spaces are on domino.lancs you will NOT be able to logon until the tests are completed, probably late Tuesday afternoon. That includes the following departments' LUVLE Spaces: Applied Social Science, Educational Research, European Languages, English, Engineering, ICT Focus, Law, LICA, Linguistics, Mathematics, Politics, Philosophy and Religion, Computing & Communications and Sociology.
Monday, 11 April 2011 - AM
This is our biggest and most important virtualisation day. As mentioned below in Friday's update, we are moving both the LUVLE Spaces from the eleven listed departments and MyModules. Remember that MyModules is the way that most users log into LUVLE, and it will be turned off until we can complete the server change. That's why we chose this between-terms day to make this change. We expect to have the server out of service -- thus unavailable -- today and tomorrow. We will announce here when it is up and running.
Friday, 8 April 2011
We've been checking on our new virtualised luvle server and all indications are that the virtualisation and transfer of LUVLE Spaces has gone well.
Next week we start on domino and ltg-domino servers, which hosts LUVLE Spaces from Applied Social Science, Educational Research, European Languages, English, Engineering, ICT Focus, Law, LICA, Linguistics, Mathematics, Politics, Philosophy and Religion, Computing & Communications and Sociology. It also hosts MyModules, which most students use to logon into LUVLE.
Thursday, 7 April 2011
This was a good day in terms of the 28-step virtualisation process. The last steps were finished and there were no problems in this process, and we have de-commissioned the old luvle server. All LUVLE Spaces appear to be functioning well; if yours is not, please let the Service Desk know.
Wednesday, 6 April 2011
After a day of monitoring, we are now convinced that the MyPlace and MyPGR services are functioning well on their new virtualised server (see the last paragraph of today's entry to understand virtualisation a little bit more).
Today we started on the luvle and ltg-luvle servers! The affected LUVLE Spaces are from LEC, Psychology, Physics, Marketing, Management Science, OWT, History, Accounting & Finance, Economics, Entrepreneurship, Management and Organisation, CEEC, and Health Research. We had to make these two servers unavailable to the network while we worked on them, but were able to minimize the time unavailable to only 6 hours and 49 minutes.
The net effect is that we have moved about 42% of the LUVLE Spaces to a virtualised server called luvle.lancs.ac.uk. Most probably you will not even notice the change, and your saved links to your LUVLE Spaces will be automatically redirected.
Someone asked me why we are bothering to virtualise servers, especially if the users will not notice anything. Here's a brief explanation.
1. The new virtualised servers are all more energy efficient than the stand-alone servers they replace. This decreases our energy consumption for both running and for cooling the server room. Think GREEN!
2. The new virtualised servers take less space in the server room.
3. We are now able to add resources, such as CPUs (sometimes called blades) and RAM, in about one tenth the time, and can dynamically react to demand a little easier.
4. The old hardware was coming to an end of its expected life span.
Tuesday, 5 April 2011
The move to virtualised servers is going well, but as we mentioned below on 16 and 30 March and in our MOTD on 1 April, the overall LUVLE service will be affected by the moves until the end of next week. We are doing further testing on MyPlace and MyPGR, but so far so good!
Monday 4th April 2011
During the relocation of the LUSI server on Saturday, MyModules was operating slowly and not displaying timetable information for about one hour. This happened again on Monday between about 10 am and 3 pm.
MyPlace and MyPGR have been moved to new hardware today, services were unavailable for about three hours.
Wednesday, 30 March 2011
Five LUVLE servers are being moved to new hardware over the next two weeks. Unfortunately there inevitably has to be some time when servers will be unavailable.
ISS apologises for this inconvenience and will do all we can to minimise the break in service. Servers are moving from dedicated systems to the ISS resilient, virtual server infrastructure already used by many other campus systems, including some other LUVLE servers.