Skip to content

Category: Configuration

How to restore files from old 8mm Exabyte tapes

8mm Exabyte backup tapes written between 1994 and 2003 were created with the ufsdump command on a Solaris box. We no longer have the Solaris box, but the Exabyte tape drives are now hooked up to algol (running linux) and can be used to recover the files using restore, the linux port of ufsrestore.

Unlike ufsrestore, restore requires you to specify the blocking factor used to write the files to tape. And, since the dump scripts wrote multiple file partitions to each tape, you also need to tell restore which tape file the partition was written to on the tape. Both the blocking factor and tape file index can be determined by inspecting the the old dump logs stored on algol under /docs/thuban-docs/dumps/

Example:

Joel hands you an Exabyte tape with the label “2/9/95” on it, and asks you to recover a file from the “/usr3” partition that was written to that tape.

1. Login to algol as root

2. cd /docs/thuban-docs/dumps/1995/

3. View the dump log for that date (e.g. ./02.09.95.dmp.gz) using emacs or less (which run gunzip on the fly), and look for ‘/usr3’.

4. Determine the blocking factor used for that tape. Look for a line like: “DUMP: Writing 32 Kilobyte records“, and remember that value (’32 Kilobyte’ in this case). Note that historically the blocking factor changed over time, so always check the blocking factor used for the particular dump in question.

BLOCKING FACTOR = 32KB

UPDATE 2015-06-17: Older tapes (for which there are no dump logs) often used a blocking factor of 10KB.

UPDATE 2015-06-17: If you don’t know the block size, use the commands “mt -f /dev/st0 setblk 0; mt -f /dev/st0 setdensity 0x0” to set the drive to “variable mode”. This gives you lower performance (only 1 block is transferred for each SCSI command, plus Check Condition overhead). If you do this, don’t specify the -b <blocksize> argument when using the restore command.

5. Now determine the index of the tape file the partition you want to recover. Starting at the top of the dump log file, counting from ‘1’, count the number of “DUMP: DUMP IS DONE” lines until you get to the “DUMP: DUMP IS DONE” line for your partition.

In this dump log, partition ‘/usr3’ was written as the 16th tape file to the tape, so:

TAPE FILE INDEX = 16

6. Insert the tape in the Exabyte tape drive

7. Rewind the tape. Currently the Exabyte tape drive is /dev/st0, so issue this command:

mt -f /dev/st0 rewind

8. Tell linux what the tape density is using ‘mt‘. Linux seems to autodetect this, but just to be safe… :

mt -f /dev/st0 setdensity 0x15

If you don’t know the tape density, you can set the density to 0 instead:

mt -f /dev/st0 setdensity 0

9. Tell linux which blocksize to use with ‘mt’. In this case our BLOCKING FACTOR is 32kb, which the ‘mt’ command likes to get as an actual byte count. So we multiply 32 x 1024 to get 32768.

mt -f /dev/st0 setblk 32768

If you don’t know the blocksize, set it to 0 (and don’t specify the -b <blocksize> argument to restore:

mt -f /dev/st0 setblk 0

10. cd to the directory you want to put the recovered files in.

cd /tmp/recovery/1995-02-09

11. Call the ‘restore’ command with the following syntax:

restore -a -i -v -b <blocksize> -s <tape file index> -f /dev/st<tape drive ID>

The above arguments tell ‘restore‘ that we want to recover files in interactive mode (-i), ignore volumes (-a), and run in verbose mode (-v).

Given our BLOCKING FACTOR of 32kb and TAPE FILE INDEX of 16, we issue this command:

restore -a -i -v -b 32 -s 16 -f /dev/st0

restore‘ should now forward the tape to the 16th tape file and then enter interactive mode, which is somewhat ftp-ish.

Once in interactive mode you can use ‘cd‘ and ‘ls‘ to explore the partition. You can specify the files to extract by issuing the ‘add <fname>‘ command. When <fname> is a directory ‘add‘ recursively targets the directory and it’s contents. The ‘ls‘ command prefixes the names of files ‘add‘ed to the extraction list with an ‘*’.

For a full list of commands use the ‘help‘ command.

Once you’ve told ‘recover‘ which files to recover to your local directory with ‘add‘ commands, issue the ‘extract‘ command to initiate the recovery. This can take several minutes, up to an hour in some cases, so go do something else and come back later.

When ‘recover‘ has recovered the files it will ask you:

set owner/mode for '.'? [yn]

Answer ‘n‘.

Here’s a sample interactive recovery session. In this case we restore the directory ‘./applications/p1391/’ from the ‘/usr3’ partition saved to the ‘2/9/95’ dump tape:

[root@algol /tmp/recovery/1995-02-09]$ restore -a -i -v -b 32 -s 16 -f /dev/st0
restore > ls
.:
11.29.dmp applications/ old_lang/ spool/
aipsnewest/ lang/ software/ tmp/

restore > cd applications/
restore > ls
./applications:
book/ games/ hiabs/ psrcat/ tempo/
ftptool/ graphing/ p1391/ starlink/ timing/


restore > add p1391
restore > extract

… [WAIT UP TO AN HOUR] …

set owner/mode for '.'? [yn] n
restore > quit
[root@algol
/tmp/recovery/1995-02-09]$

Kudos to James Fuller for figuring this out.

konqueror (nautilus?) file browser and kwrite (kate?)text editor setup hints

Added by jmw 2020 may 28: Maybe these konqueror and kwrite config hints are still somewhat useful for nautilus and kate???

I find konqueror the best file browser for linux. To customize it a little, for example to get .idl files to open with my favorite text editor, kwrite:

with konqueror running:
settings>>configure konqueror gives some options like what your home dir should be (eg /home/youusername

to make conqueror behave a certain way with a certain kind of file, i made in my home dir (/home/jweisber) a file called dummy.idl

if you select it inside konqueror (one click to highlight it)

go to edit>properties
it should pop open a window labelled “Properties for dummy.idl – Konqueror”
‘way to the right of “type” is an icon of a bunch of tools. push on it.
it should pop open a window labelled “Edit file type application/idl KDEditFileType- Konqueror”
if its not already there, add a filename pattern of *.idl
that means any file ending with that will be treated in the way you are getting ready to set up

under application preference order, enter
kwrite

if you wish, you can also associate a pretty icon with *.idl by clicking on the “?” icon to left of “filename patterns”

click ok and from now on , when you dble-click on any file ending in .idl, it should open it with kwrite (see addition below)

(you may have to entirely quit and restart konqueror for the new properties to take)

–jmw january 2008

——addition feb 2008 – to get kwrite to use “syntax highlighting” (ie to color idl files with a certain color set; fortran etc etc):

open an idl file in kwrite.

in the kwrite toolbar up top, hit settings>configure editor>highlighting

now here is what took me more time than i am willing to admit:

the panel should now be labelled highlighting rules.

the first thing to do is to select a type from the pulldown menu near the top; eg “sources idl ”

then inside properties:file extensions, type *.pro;*.idl;*.batch

inside properties: MIME types, type application/idl

then hit the icon on the far right of this line and get a panel saying “Select the mimetypes you want highlighted using sources/idl syntax highlight rules”

and check the relevant mime types.

(note if the right mime types arent there, see joel)

note that some of these properties do not “take” until you jump out of konqueror and restart it!

Algol can now talk to its SDLT Tape Drive

Algol has never been able to properly communicate with the SDLT tape drive. The symptoms were that you could write/read single small file to/from the tape drive, but an attempt to write a *directory* to it drive would fail with this message in shell:

tar: /dev/st1: Cannot write: Input/output error
tar: Error is not recoverable: exiting now

…and this message in /var/log/messages:

Jan 9 15:20:08 algol kernel: st1: Error with sense data: Current st1: sense key Aborted Command
Jan 9 15:20:08 algol kernel: Additional sense: Synchronous data transfer error

The first thing that James Fuller and I did was to see if the other tape drive, a sun 8mm exabyte drive known to algol as /dev/st0, was working. It worked fine. That indicated that algol’s Adaptec 39160 scsi controller card was as least partly working.

The next thing we did was reboot algol. This wasn’t really planned as part of our attempt to get the tape drive working, but rather to test the newly installed idl7.0 license manager. But it was a good thing we did. During the boot sequence we saw a screen titled “Adaptec SCSI Select: hit Ctrl-A to enter”. Prior to this time we had no idea such a utility existed. It’s basically a BIOS for the scsi controller card.

We hit ctrl-a, entered the utility and poked around. The utility enables you to see, for each of the two “channels” A & B (ports), the devices mapped to device ID’s 0-15. On channel A, only the scsi controller itself is mapped (to device ID 7). On channel B, the exabyte tape drive was mapped to device id 0, and the Quantum SDLT was mapped to device ID 6 and the scsi controlloer to device ID 7.

Each channel screen showed a matrix of the devices and parms for each device. The first such parm was “Initiate Wide Negotiation”. This rang a bell in James’ head — he remembered reading somewhere that this parm, when turned on, could cause problems for some scsi devices. So we turned it off for device 6. Note: making this change had the side effect of changing the communications speed value from 160Mb/s to 40Mb/s.

However, when we exited the utility and allowed the reboot to complete, we could write to/read from the SDLT tape drive.

I verified that it was really working by writing /home to it, reading it back to a temporary location, and then doing a recursive diff (diff -rq /home /root/foodir/home) against the original and the copy read from the tape. There were no diffs, and /home takes up 1.5G so I feel confident that Algol can really talk to the SDLT tape drive now.

sudoer list

As of this post date 2007, the Astro network syncs thuban’s sudoers file to the other workstations, just as it does with user accounts. Please contact fullerj with any problems regarding sudo.

Where to set up Environment variables etc for ALL users on login

OUT OF DATE as of 2008-11-01.

Here is how to set the path or add to the path for all users when they log in. The procedure depends on whether the users’ shell is tcsh (usual case) or bash.  (To determine if your shell is tcsh, type ‘printenv SHELL’.)

———————

TCSH:

For tcsh users, the file to modify is /etc/csh.login. (Also note that other startup conditions can be set for all users in /etc/csh.cshrc).  To add an environment variable simply type

setenv JAVA_HOME /usr/java/1.3.1

To append to the Path in say YOUR OWN .cshrc, type:

setenv PATH “${PATH}:/path/to/be/added”

(This will append your new desired path to the path set for everyone in /etc/csh.login)

————————————————-

BASH:

For users that like bashrc, you can modify /etc/bashrc to add the necessary path or variable. To add a variable, for instance java’s home directory, simply type

export JAVA_HOME=”/usr/java/jdk1.3.1″

If you wanted to add an extra path to sys set up path in YOUR OWN bashrc, type:

export PATH=$PATH:path/to/be/added

Date and Time Configuration

OUT OF DATE as of 2008-11-01.

For our systems, we want to use the network time protocol put in place by the college. Therefore, we do not set the time manually but rather, have it sync from the Carleton Network Time server. To do this follow these easy steps.

As root, under the Applications menu to the right of the little red hat, click on System Settings/Date and Time.

Click on the Network Time protocol tab. Under this tab you should see a place to add and delete network time servers. Delete any servers that are there.

Then type ntp.carleton.edu and click the add button. ntp.carleton.edu should then appear as a server on the list. Click OK and wait a few moments for the machine to sync with the server.

Double check the time to make sure it is accurate. Your system should now have the correct date and time.

Algol tape drives

We have two tape drives now attached to Algol. The first is an exabyte tape drive listed under

/dev/st0 (or /dev/nst0 depending on if you are using no rewind or not)

The SDLT 320 tape drive is listed under /dev/st1 (/dev/nst1).

To read data from the exabyte. Simply CD into the directory you want to put your data in. Then use the command:(must be run as root)
>>tar xvf /dev/st0

If this fails, try setting the bock size and density using these two commands and then try again.

>>mt setdensity 0x00

>>mt setblk 0

To extract from the SDLT tape drive, the tar command should work as well.

WARNING: When retrieving data from tape, make sure the directory you are in is empty as you can overwrite data files without even realizing it.

Configure astro network client

OUT OF DATE as of 2008-11-01.

OUT OF DATE as of 2008-11-01.

Steps to configure an astronomy department linux client computer.

  1. Install Redhat Enterprise Linux AS 4
  2. Install redhat keys from:
    http://rhn.carleton.edu/pub/RedHat/keys.html
  3. Add line to /etc/hosts
    137.22.6.54 algol.physics.carleton.edu algol
  4. Add to /etc/fstab:
    algol:/home /home nfs defaults 1 1
    algol:/etc/secret /etc/secret nfs defaults 1 1
  5. Add computer’s IP address to Algol’s /etc/exports file and then run:
    /usr/sbin/exportfs -a
    on Algol
  6. Install Carleton RPM for LDAP auth:
    rpm -i –force carletonldapauth-1.00-3.es40+fc2.noarch.rpm
  7. Put mntdrives and umntdrives scripts in /usr/local/bin available on /etc/secret/RPMS/mntdrive_scripts (mntdrives.tar.gz)
  8. Make collab scripts runnable:
    chmod u=rwxs,g=rxs,o=rxs /usr/bin/smbmnt
    chmod u=rwxs,g=rxs,o=rxs /usr/bin/smbumount
  9. cp /etc/secret/RPMS/cron_scripts/redhat_update.cron /etc/cron.daily
  10. /etc/cron.daily/redhat_update.cron
  11. cp /etc/secret/RPMS/cron_scripts/get_account_info.cron /etc/cron.hourly
  12. /etc/cron.hourly/get_account_info.cron
  13. Set up printer for computers in Olin 304 and 311 (see separate “printer setup” post)
  14. test