UPGRADE-HOWTO

            Manhattan Virtual Classroom 3.3
Upgrade instructions


This document provides instructions for upgrading Manhattan from a
variety of earlier versions to the current release.

If you are in a hurry and need to upgrade your Manhattan installation as
quickly as possible, move slowly and read these instructions through completely
before carefully performing the steps. If you'd like to spend many hours on
this, then just skim this document and do what feels right given your
extensive experience with information systems.

First, determine which version of Manhattan you are currently
running. Then follow the instructions in the list below for your
version. Note that not all of the instructions in this document will
apply for you!

To find out which 2.x version you are currently running, login as
system administrator and note the version number near the bottom of
the main menu's page.

For versions 3.x, the version number is listed in the upper right
corner on the system administrator's page. You can also use
your web browser to view the HTML source of any of Manhattan's pages.
The Manhattan version number is listed in a comment at the top of
the HTML source.


If you are running version:

0.9x There is no way to upgrade to this version of Manhattan.
The 0.9x releases are vastly different from the 2.x and
3.x series. You should follow the instructions in the
file named INSTALL and start with a completely new
installation.

3.2 1) Read the notes under VERSION 3.3 CHANGES
2) Follow the BASIC UPGRADE INSTRUCTIONS


3.1, 3.0 or 2.4 1) Read the notes under VERSION 3.2 FILE FORMAT CHANGES
2) Read the notes under VERSION 3.3 CHANGES
3) Follow the BASIC UPGRADE INSTRUCTIONS


2.2 or 2.3 1) Read the notes under VERSION 3.2 FILE FORMAT CHANGES
2) Read the notes under VERSION 3.3 CHANGES
3) Follow the BASIC UPGRADE INSTRUCTIONS
4) Follow the instructions in the section
FIXING POSTOFFICE SYMLINKS


2.1 or 2.1.1 1) Read the notes under VERSION 2.2 SURVEY CHANGES
2) Read the notes under VERSION 3.2 FILE FORMAT CHANGES
3) Read the notes under VERSION 3.3 CHANGES
4) Follow the BASIC UPGRADE INSTRUCTIONS
5) Follow the instructions in the section
FIXING POSTOFFICE SYMLINKS

2.0 or 2.0.x 0) Read the section UPGRADING FROM 2.0.x
to understand the upgrade process.
1) Follow the BASIC UPGRADE INSTRUCTIONS
2) Follow the UPGRADING FROM 2.0.x instructions
3) Follow the instructions in the section
FIXING POSTOFFICE SYMLINKS
4) Read the notes under VERSION 3.2 FILE FORMAT CHANGES
5) Read the notes under VERSION 3.3 CHANGES





************************************
**** BASIC UPGRADE INSTRUCTIONS ****
************************************

The basic idea is that you need to compile the new release, then replace all
of the files found in your live version's sbin, bin, images, and chat
directories with the new ones. The steps below accomplish this in the
SAFEST way possible.


0) Optionally, backup your server. This really shouldn't be necessary,
since there's nothing risky about the upgrade procedure, provided
you don't do something silly like accidentally delete the wrong
directory. You might skip this step if you do regularly scheduled
nightly backups, but it's your data.

1) Get everyone off your live installation and shut down the chat
service:

a) Login to the administrative system of your live version as
super-user.

b) Find, and click the "Lock Server" command on the menu. This
prevents new users from logging in, and will kick logged-in
users out when they hit the Main Menu for any course.
Repeatedly view the "Who is on Manhattan?" page from the Admin
menu to determine for yourself when everyone is truly off the
server. (Manhattan's Admininstrator's Reference at
http://manhattan.sf.net describes this process in detail -
Read the section "Lock server".)

c) From the Administrators Main Menu, click on "Manage chat",
then click on "Stop the Melange chat service"

2) When you're sure everyone is off the system, login to a shell
account as user 'manhat' and 'cd' to your live distribution's
directory, which is called something like:

manhat-3.2.0 or
manhat-3.1.0 or
manhat-3.0.0 or
manhat-2.4 or
manhat-2.3 or
manhat-2.2 or
manhat-2.1 or
manhat-2.1.1 or
manhat-2.0 or whatever

Carefully move away the src, bin, sbin, images, and chat directories.
My suggestion is to :

mkdir old_version # create a new directory named old_version
mv src old_version
mv bin old_version
mv sbin old_version
mv chat old_version
mv images old_version

Move the various readme files into the old_version directory as well:

mv README old_version
mv CREDITS old_version
mv INSTALL old_version
mv CHANGES old_version
mv UPGRADE-HOWTO old_version

DO NOT touch the other directories, including 'users', 'courses',
and 'templates' - they hold the course data for your installation.


3) Unpack the new release's tar file somewhere on your system with:

tar -xvzf manhat.x.x.x.tar.gz

(You've probably already done this, since you are reading this file!)

The files will unpack into a new directory named manhat-x.x.x (where
x.x.x is the version number of the new release. Change to that
directory and move the 'src' directory into your live installation's
directory.

cd manhat-x.x.x # that is, change to the directory that
# was created when you unpacked the tar
# file

mv src PATH-TO-YOUR-LIVE-INSTALLATION

Replacing PATH-TO-YOUR-LIVE-INSTALLATION with the appropriate path. For
example if your live installation is in home/manhat/manhat-3.0.0 you
would type:

mv src /home/manhat/manhat-3.0.0

Repeat the process with the 'images' directory. That is, move the new
version's 'images' directory into your live installation's directory.

mv images PATH-TO-YOUR-LIVE-INSTALLATION

Move over the various README files in the same way:

mv README PATH-TO-YOUR-LIVE-INSTALLATION
mv INSTALL PATH-TO-YOUR-LIVE-INSTALLATION
mv CREDITS PATH-TO-YOUR-LIVE-INSTALLATION
mv UPGRADE-HOWTO PATH-TO-YOUR-LIVE-INSTALLATION
mv CHANGES PATH-TO-YOUR-LIVE-INSTALLATION


4) Read and follow the INSTALL instructions for the new release that
are in the file named INSTALL that you just moved over. Use these
additional notes as a guide. The steps refer to steps within the INSTALL
file:

- Start with Step 3, since you've already taken care of steps 1 and 2

- You can probably skip Step 8 (Configure Apache). Since you're
upgrading, nothing has changed with Apache configuration requirements.

- Step 9 has you run a shell script called 'set_dirs'. If you have lots
of classrooms and/or lots of users on your installation, it can take a
few minutes for this script to finish. Relax. Breathe deeply.
Everything is going well.

- D0 NOT SKIP STEP 10. This tests your installation, but it also
creates/updates the configuration for your installation.

- Skip Step 11. Your super-user username and password remains the same as
it was for your previous installation.



5) Assuming the INSTALL went well, you are now running the new version.
Login as the super-user and look around to convince yourself that
everything looks good.

If you normally run the chat service, click on "Server-->Manage Chat" and
then "Start the Melange chat service".

Check the server configuration settings under "Server-->Configure server"
to make sure the settings are the way you want them.

When you are ready to open up the system for
students and teachers, select "Server --> Unlock server".



If for some reason you can't get the new Manhattan to install, you can
revert to your old installation by moving the bin, sbin, images, and
chat directories back from the old_version directory where you put them
in step #2 above. For many upgrades, it is important for you to do this
BEFORE you unlock the server. This is because for some upgrades, changes
occur to a users account as they login and use the system. See the
recommended sections for your upgrade to see if this applies to you.

# End of BASIC UPGRADE INSTRUCTIONS #






************************************
**** VERSION 3.3 CHANGES ***********
************************************

Version 3.3 introduced a few system level changes that you should know
about. The bottom line is that once students and teachers begin using your
system after upgrading from a version earlier than 3.3 you cannot (easily)
revert back to an earlier version.

'id.prefs' replaces 'id' files
------------------------------
In previous versions, a text file named 'id' was used to store the full
Real Name, e.g. 'John Smith' and ID number of each user on the system. Each
user had their own 'id' file, which was stored in their home directory under
Manhattan's 'users' directory. Starting with version 3.3 the old 'id' file
is replaced by a file named 'pref.hdf'. This text file is in a Hierarchal
Data Format (hence the 'hdf' extension), that can be quickly read by the
ClearSilver library (http://www.clearsilver.net) used by Manhattan. Moving
forward, this 'prefs.hdf' file can be used to store any 'preferences' that
apply to a particular user.

After upgrading from a version earlier than 3.3, whenever Manhattan
encounters an old-style 'id' file, it automatically replaces it with an
equivalent new-style 'prefs.hdf' file. You cannot switch back to an earlier
version because those programs expect to find the older 'id' files for each
user.


'custom.h' no longer used
-------------------------

Another significant change starting with version 3.3 is the elimination of
the 'custom.h' file. The 'custom.h' file contained a list of server
configuration settings. It was always recommended that you check the
'custom.h' file before compiling or building Manhattan when it was installed
or upgraded. To change a configuration item, you had to edit 'custom.h'
then rebuild the entire system. The 'custom.h' file in no longer used.
Instead, the configurable items that were previously stored in 'custom.h'
are now changeable on the fly via a 'Server --> Configure Server' item on
the Administrator's menu.


When upgrading from version 3.2 or earlier, you should locate the 'custom.h'
file located in the 'src' directory of the version you are upgrading from.
Open this text file in a text editor. The values in this 'custom.h'
file correspond directly to the server parameters now accessible from the
administrators web-based system under 'Server --> Configure server'. By
comparing your old 'custom.h' file to your new installation's configuration
parameters, you can ensure your new installation will behave the way you
want it to.


Fewer directories used
---------------------

If you have ever examined your installation's 'courses' directory, you
know Manhattan uses lots of directories (folders) to keep things organized.
In practice, many of these directories were left empty in earlier versions,
for example because a particular module was never used in the classroom.
The new version uses the same directory structure as for pre-3.3 versions,
but it creates these directories on an as-needed basis. The result is
Manhattan versions 3.3 and later will create many fewer directories on your
file system.

After you have upgraded from a pre-3.3 version, you can run a utility called
'rm_empty_dirs' to delete all of the empty directories used by your older
courses that are no longer needed by the new version of Manhattan:

1) change to the newly installed 'src' directory
2) type the command:

make rm_empty_dirs

3) run the program by typing:

./rm_empty_dirs

You'll get a screenful of instructions, but no action will be taken unless
you run the program with the appropriate command line 'switch'.

NOTE: running this 'rm_empty_dirs' program is completely optional. You may
want to leave this step until after your new installation has been up and
running for awhile.






************************************
**** VERSION 3.2 FILE FORMAT CHANGES
************************************

Version 3.2 introduced the ability for teachers and students to mark
messages with a 'flag'. This provides a simple method for marking
messages of interest they might want to return to later.

A necessary data conversion is done automatically on-the-fly as
teachers and students begin to use their classrooms after you upgrade
to the current version.

Because of this, once teachers and students begin to use their
classrooms running this version, you can't revert back to a previous
version. (I don't know why you'd want to. The only interesting
direction is ahead!)

Note this non-reversible data conversion is not done to all classrooms
the moment the new version is installed. The conversion happens when
each user enters a particular module in a particular classroom. You
CAN revert to an earlier version IF no student or teacher has logged
in.









*************************************
**** FIXING POSTOFFICE SYMLINKS ****
*************************************

(This section will be of interest to you if you are running any 2.x
version prior to 2.4)

Manhattan 2.4 introduced the ability for users to permanently delete
Post Office messages. The Post Office module allows users to send each
other private messages, that can contain file attachments. Since the
first version of Manhattan, users were unable to delete Post Office
messages by design. The thought was that if it were absolutely
impossible for the sender or recipient to delete a Post Office message,
it would also be completely impossible for someone to lie about whether
or not they received or read a Post Office message.

When you sent someone a Post Office message, they got it - and you know
it, and they know it because it's still there! Still, it's difficult to
explain to people that this is a feature and not a flaw. So starting
with version 2.4, students and teachers can delete Post Office messages
as if they were ordinary Internet e-mail messages.

This new ability raises a potential problem that needs to be fixed.
While the fix is as simple as running an upgrade program, explaining
why it needs to be done takes a few words.

Say you sent a Post Office message to Jim, which contained an attached
file. Jim decides to "Forward" the message, along with the attached
file, to Judy. (He does this by clicking on the Forward button while
reading the message.) To save disk space, Manhattan keeps just one copy
of the attached file. In versions prior to 2.4 'symbolic links' to the
original file were created when the file was forwarded.

- You and Jim see the original file
- When the message is forwarded to Judy, she sees a symbolic link
to the original file.

original_file
Judys_symbolic_link --> original file

This process of creating symbolic links (which are files that point to
other files) can continue indefinitely. Judy could forward the file to
John, who could forward it to Michael, and so on:

original_file (seen by the original sender and recipient)
Judys_symbolic_link ---> original_file
Johns_symbolic_link --> Judys_symbolic_link
Michaels_symbolic_link --> Johns_symbolic_link

This worked great in the past, except now its possible for someone to
delete a Post Office message. Suppose you, the sender of the first
message containing original_file, delete the message from your Post
Office Outbox. Then Jim, the recipient of the message containing
original_file, deletes the message from his Inbox. Manhattan detects
that everyone involved with that original Post Office message has
deleted the message, so it will permanently delete original_file. The
problem is, once that's done, any and all symbolic links created by
forwarding the file become 'broken'. Judy, John, and Michael would
get error messages when they attempt to view the file, which was
forwarded to them.

The 'fix_postoffice_symlinks' program simply searches all courses for
the presence of symbolic links to files created by the Post Office,
and replaces them with a 'hard link'. (Hard links save disk space
like symbolic links, but they don't have the problem of 'breaking' as
described above.)

1) First, compile the program:

cd to the 'src' directory of your upgraded installation and type:

make fix_postoffice_symlinks

2) Now run the program with:

./fix_postoffice_symlinks

The program runs without any prompts and ends by reporting the number
of symbolic links it processed.



### End FIXING POSTOFFICE SYMLINKS ###





*************************************
**** VERSION 2.2 SURVEY CHANGES ****
*************************************

(This section will be of interest for you only if you are upgrading
from version 2.1 or 2.1.1)

Since version 2.2, Manhattan handles survey data differently from
versions 2.1 and 2.1.1 in order to support a full-fledged Survey
module accessible to teachers within a Manhattan classroom. (There
was a completely undocumented survey capability available only to
system administrators in 2.1.x releases.)

If you have issued surveys under one of these previous releases, THAT
SURVEY DATA WILL BE INACCESSIBLE once you upgrade to this version If
you have any outstanding surveys that you are waiting for
students/teachers to respond to, you should WAIT until the surveying
process is complete before you upgrade.

If you have used the survey capability of the earlier 2.1 or 2.1.1
releases, use the Surveys area of the Administrative system to view
results of the surveys, and the surveys themselves, and print them
out or download them to your PC before you perform this upgrade.

ALL SURVEYS created with earlier versions will seem to DISAPPEAR when
you upgrade.

If you have created a large number (or a few complicated) surveys
using the previous version of Manhattan, and can't bear the thought
of re-typing them, send me an email, and I'll work with you to get
them moved to the new system. It IS possible to re-use the surveys
in the new release, but an explanation of how to do so would be long
and I suspect it would only benefit a few people.

After upgrading, old survey data is NOT deleted. Instead, it is
ignored (or 'orphaned'). In previous releases of Manhattan survey
data was stored in the directory:

manhat-x.x.x/users/admin/survey

After upgrading you can move or delete this directory, since it is no
longer used.


# End of VERSION 2.2 SURVEY CHANGES #




*****************************
*** UPGRADING FROM 2.0.x ****
*****************************

IMPORTANT!!!!!!
Read this at least TWICE before you perform this procedure!

These instructions are ONLY for upgrading from:

version 2.0 or
version 2.0.1


==============================================
What does the upgrade do?
==============================================

In versions 2.0 and 2.0.1 of Manhattan, all of the data for
courses on your server were organized as a flat list of
directories within your installation's 'courses' directory.

cd YOUR_CURRENT_INSTALLATION/courses
ls
s03cis10202 <--- each directory holds data for one
s03cis10204 course.
s03cis10208
s03cis10210
s03cis20202
...


Each directory listed represents one Manhattan course in
your old installation. The problem with this organization
is that as the number of courses on your server grows, it
gets more difficult to manage as an administrator. The new
version of Manhattan addresses this by organizing the
courses in SUBDIRECTORIES (called course groups) under the
'courses' directory.


Sample 'new' courses directory - organized into
subdirectories:

f03 <-- a subdirectory to hold "Fall 2003" courses
phy10101
engl34002 <-- directory for each Fall 2003 course
...
...
w03 <--- a subdirectory to hold "Winter 2003" courses
phy10303
engl23202 <-- directory for each Winter 2003 course.
...
demo <-- a subdirectory to hold 'demo' courses
demo101
demo102 <-- a directory for each 'demo' course
...

The new code assumes the courses are organized
into subdirectories. The upgrade program will move all of
your current course directories into a SINGLE SUBDIRECTORY
named 'orig' under the courses directory. At the same time it
adjusts each user's profile to reflect the new path to the
courses that user belongs to.

The second change regards login logs, which are stored as
text files in the

users/logs

directory of your current installation. The new version has
additional logging features, but this required a change in
the format of the log files. The upgrade program moves
your old log files to the directory

users/old_logs

and starts completely new login logs. You can use a text
editor to view the old logs after the upgrade, or you can
delete them if you wish.


The third change involves how a Manhattan administrator's
access rights are controlled. The old version used an XML
file format, stored in, for example:

/users/admin/admin_conf.xml

While using XML seemed like a good idea at the time, it
makes it more difficult to add new features to the admin
menu (the one you see as super-user) in the future. After
upgrading, you'll have to create an entirely new
super-user account for your installation. If you had other
administrators on the system, you will have to recreate
those accounts as well.

======================================================
Can I go back to the original version after the upgrade?
=======================================================

The short answer is 'no'. However, before you start the
upgrade you should definitely do a complete backup of your
existing installation. If some unforseen event (unlikely)
occurs during the upgrade, you can restore the original
installation from the backup.


=======================================================
What's the upgrade procedure?
======================================================

1) Follow the BASIC UPGRADE INSTRUCTIONS in this document.
However, don't bother trying to login and test anything.
You won't be able to login as an admin, teacher, or
student until you run an upgrade program.

2) Compile the upgrade_from_2.0 program:

From the src directory, type:

make upgrade_from_2.0

This will compile the upgrade program's source code
into an executable file.

3) Run the upgrade program:

From the src directory, type:

./upgrade_from_2.0

to run the upgrade program, and follow the prompts.


4) Create a new super_user account:

Use a web browser to visit:

http(s)://YOURSERVER.edu/manhat2-sbin/super_conf

Follow the instructions to create a new super-user
account.


Login as super-user and unlock the Manhattan server.

Don't forget to check the list at the top of this file
for any additional steps that might be required.


### End of UPGRADING FROM 2.0.x ####