Carleton Astronet Admin Log – Page 10 – Carleton Astronet Admin Log

Use algol:/root/tapebackup/backup-thuban-to-magtape.sh to create tape backups of Thuban

Published by jweisber on 2010-03-12

To create a tape backup of Thuban, insert an SDLT tape into Algol’s SDLT tape drive, ssh into Algol as root, and run /root/tapebackup/backup-thuban-to-magtape.sh.

This script writes a log file into /root/tapebackup that should be reviewed for any problems that tar might have encountered during the backup process. See the script for details.

Assuming the backup tape is written successfully (check the log file), make sure it’s actually readable by running /root/tapebackup/tar-tvf-magtape.sh. This script will prompt you for a date in ‘YYYY-MM-DD’ format (supply the date the tape was written) and will then perform a ‘tar -tvf’ on the tape, writing stdout to tar-tvf-YYYY-MM-DD.log and stderr to tar-tvf-YYYY-MM-DD.err. If the tar tvf command completes successfully, and the .err file is empty then we can have some confidence the data on the tape is readable — and we have an searchable index of the tapes contents.

For example, if you want to know what ‘.sh’ files exist in any subdirectory of /data/psrdata/usr5/1913gr/’ in any of the backup tapes, you can cd to algol:/root/tapebackup and issue the following command (note the lack of leading ‘/’, which is not present in the ‘tar -tvf’ filepaths):

grep 'data/psrdata/usr5/1913gr/' tar-tvf*.log | grep '\.sh'

…which will generate a listing like this:

tar-tvf-2010-07-01.log:-rwxrwxr-x jweisber/pulsar 3017 2010-04-06 15:08:50 data/psrdata/usr5/1913gr/tempo2010/tempoloop-gl1.sh tar-tvf-2010-07-01.log:-rwxrwxr-x jweisber/pulsar 2801 2009-12-09 14:08:08 data/psrdata/usr5/1913gr/tempo2008/tempoloop-r-s.sh tar-tvf-2010-07-01.log:-rwxrwx--- jweisber/pulsar 2256 2009-08-20 01:38:51 data/psrdata/usr5/1913gr/tempo2008/tempoloopfordavid.sh tar-tvf-2010-07-01.log:-rwxrwxr-x jweisber/pulsar 3017 2010-02-16 09:33:16 data/psrdata/usr5/1913gr/tempo2008/tempoloop-gl1.sh tar-tvf-2010-07-01.log:-rwxrwxr-x jweisber/pulsar 3017 2010-03-21 17:27:19 data/psrdata/usr5/1913gr/tempo2008/tempoloop-gl2.sh

Must supply ‘-P’ arg when extracting files from Thuban backup SDLT tape

Published by jweisber on 2010-03-12

When restoring files from a Thuban SDLT Backup tape (via Algol’s SDLT tape drive) you must specify the ‘-P’ argument’ to tar, like so:

tar -xvPf /dev/nst1 $RESTOREFILE

If '-P' is not specified, a random set of symlinks in the restored file set will end up being zero length regular files rather than symlinks.

Note that even when you supply the ‘-P’ option, the original date on the symlink is not restored, but is given the current date instead.

This bug doesn’t seem to manifest for trivially small backups, but definitely manifests in our Thuban backups.

This bug is apparently due to a race condition that occurs while tar is restoring the symlink.

See this website for a more in-depth explanation.

Quota Check

Published by jweisber on 2010-01-22

Added a script to the bashrc, zshrc, and csh.cshrc files that automatically checks if you are over your quota.

Every time you open a new terminal, run an xterm, or ssh into a computer it should echo “Quotacheck OK (/home/$USER is <used/softcap>% full)” if you are under quota. If you are over quota it will tell you, so either delete files or talk to Joel about increasing your quota size.

idl homemade plot symbols

Published by jweisber on 2009-11-24

UPDATE 2012 MAY JMW: SYMBOL is now included as part of IDL. By the way here is the calling sequence:
PLOT,X,Y,PSYM=SYM(NUMBER)

————————————————————–

IDL has a poor selection of plot symbols, so I downloaded routines from the web that do a better job. Look at symbol.pro in /usr/share/astro/idlshare/local.

I should somehow put this in a library but for now I copied and pasted the routines (except the first, driver routine) into the top of a user file and then called it to get nice fat filled circles.

See /data/psrdata/swigglitch/GlitchRead.4.1.pro

Install Mathematica 7

Published by jweisber on 2009-11-13

Put in Mathematica 7 CD, ensuring that it appeared on the desktop.
Open a terminal window and cd to /media/Mathematica/Unix/Installer
Run ./MathInstaller.
(For some reason the Mathematica Installer won’t open unless you run it from the terminal)
Note, if it throws an error that says “/bin/sh: bad interpreter” point the shell directly at MathInstaller by typing:
/bin/bash ./MathInstaller

Choose default install directory:

/usr/local/Wolfram/Mathematica/7.0/

Allow selinux modification

Choose default script directory:

/usr/local/bin

For password configuration choose option (1):

(1) Single machine
Install a password specific to this machine. Mathematica will launch,
and you can enter your password.

This is what the Installer does upon selection of option (1).

xxxxxxxxxxxxxxxx
Configuring Single-Machine password…

Mathematica 7.0 for Linux x86 (32-bit)
Copyright 1988-2009 Wolfram Research, Inc.

You will need to get a password from your
license certificate or from Wolfram Research
(http://register.wolfram.com).
Machine name: antigone.Physics.Carleton.edu
MathID: 7108-71426-20286

You will need a valid license ID and password in order
to proceed. Go to http://register.wolfram.com or
consult your Getting Started documentation.
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Then, go to (http://register.wolfram.com) and type in the information the Installer gave you; Machine Name, MathID, and Carleton’s License (L3125-8812). The registration web page will ask for the name of the person registering, supply either Joel Weisberg or Nelson Christensen, but give Bruce’s email address (or your own email, just forward the forthcoming email to Bruce).
So it will look like:

Name: Joel Weisberg
Organization: Carleton College
Department: Physics and Astronomy
Email address: bduffy@carleton.edu

This will cause an email to be sent the email address you entered with the password needed to finish the installation.

Here are the contents of the email I received:

xxxxxxxxxxxxxxxxxxxxxx
Version 7.0 password(s) were generated for license: L3125-8812

Here is the content for your mathpass file:

antigone.Physics.Carleton.edu
7108-71426-20286
L3125-8812
5157-617-135:2:8:20080701

Requestor information:

Name: Joel Weisberg
Organization: Carleton College
Department: Physics and Astronomy
Email address: bduffy@carleton.edu

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

The install script then asked for:

Carleton’s License (L3125-8812)
The Requestor (Nelson Christensen/Joel Weisberg)
The organization (Carleton College)

Then the Installer asked for the password (the one from the email I received was 5157-617-135:2:8:20080701).

The password will be the last string of digits given in the email. (And the only string you didn’t have before the email)

idl71 install notes

Published by jweisber on 2009-09-01

==========================================================
IDL71 Install instructions
==========================================================

NOTE: These files are stashed in /etc/secret/idl71:

idl71-install-notes.txt
idl71-license.dat
astron.tar.gz

# cd /usr/local # mv itt itt70 # mkdir itt # cd itt

As the installation booklet suggested might happen, my attempt to run the install script failed with:

[root@algol itt]# /media/IDL71/install_unix.sh bash: /media/IDL71/install_unix.sh: /bin/sh: bad interpreter: Permission denied
I applied the remedy they suggested:
[root@algol itt]# umount /dev/hdb [root@algol itt]# mount -o ro -t udf /dev/hdb /media
NOTE: For algol, the device was ‘/dev/hdb’, but for sirius the device was ‘/dev/scd0’:
[root@Sirius itt]# umount /dev/scd0 [root@Sirius itt]# mount -o ro -t udf /dev/scd0 /media
… and then ran the installer again:
[root@algol itt]# /media/install_unix.sh
The installer launched and asked me what I wanted to install. I chose only IDL program files and skipped the “CIA High Res Maps” and “Dicom Network Services” options.

After install, a ‘Product Environment’ window pops up. I chose the ‘Create Links’ option.

The “Licensing/Registration Program” window pops up. I chose to run the license wizard, but it seems as if SELinux kept it from popping up. Instead, the installer finished by giving me the full path the the license wizard so that I could run it later:

/usr/local/itt/idl71/bin/ittlicense

When I launched the license wizard by hand I got SELinux warnings:
[root@algol itt]# /usr/local/itt/idl71/bin/ittlicense /usr/local/itt/idl71/bin/bin.linux.x86/idl: error while loading shared libraries: /usr/local/itt/idl71/bin/bin.linux.x86/libidl.so.7.1: cannot restore segment prot after reloc: Permission denied rm: cannot remove `/root/.ittlicense_tmp': No such file or directory [root@algol itt]#
The SELinux troubleshoot browser posted an alert, and specified the commnands necessary to make SELinux shut up. I issued those commands, but the wizard posted a window complaining about another SElinux denial… At this point I changed the systems SELinux state from ‘Enforcing” to “Permissive”…

I pasted the license.dat info into the text typin and clicked through all the options (install license mgr, launch license mgr) and then select Exit.

NOTE: the idl71 license wizard makes it’s own provisions for starting the license manager at boot time. So I removed the old idl70 specific license manager launching command from /etc/rc.d/rc.local:

/usr/local/itt/idl70/bin/lmgrd -c /usr/local/itt/license/license.dat

NOTE: Must also kill the running lmgrd process for the previous version.

Made sure all current users ~/.cshrc file contained the correct idl_setup call. They did. Here’s the proper command:

source /usr/local/itt/idl/bin/idl_setup

Installed the goddard lib:

mkdir /usr/local/itt/idl70/lib/goddard cd /usr/local/itt/idl70/lib/goddard wget http://idlastro.gsfc.nasa.gov/ftp/astron.tar.gz tar -xvzf ./astron.tar.gz chown root:users -R .

I then rebooted, logged in as bduffy, ran a csh shell and issued the idlde command. Seemed to work….

============================================================
Install issues on Sirius and Mirzam
============================================================

On Sirius (and Mirzam) the installation experience was different. After setting SELinux to Permissive mode, I launched the license wizard and it it segfaulted.

% Embedded IDL: ITT Visual Information Solutions, License Wizard.
% Embedded IDL: ITT Visual Information Solutions, License Wizard.
% Loaded DLM: XML.
/usr/local/itt/idl71/bin/ittlicense: line 36: 27919 Segmentation fault $IDL_DIR/bin/idl $* -em=$IDL_DIR/lib/hook/ittlicense.sav
[root@Sirius itt]#

NOTE: I could launch idlde, but it complained about the missing license manager…

SOLUTION:

I could see on algol how the license manager was launched. There’s a file in /etc/init.d/sys5_idl_lmgrd and symlinks to it in /etc/rc.d/rc*.d. So:

From algol:

rsync -av /etc/init.d/sys5_idl_lmgrd root@sirius:/etc/init.d/

From Sirius:

ln -s /etc/init.d/sys5_idl_lmgrd /etc/rc.d/rc0.d/K01sys5_idl_lmgrd ln -s /etc/init.d/sys5_idl_lmgrd /etc/rc.d/rc2.d/S99sys5_idl_lmgrd ln -s /etc/init.d/sys5_idl_lmgrd /etc/rc.d/rc3.d/S99sys5_idl_lmgrd ln -s /etc/init.d/sys5_idl_lmgrd /etc/rc.d/rc4.d/S99sys5_idl_lmgrd ln -s /etc/init.d/sys5_idl_lmgrd /etc/rc.d/rc5.d/S99sys5_idl_lmgrd

I then rebooted, saw lmgrd start properly at boot, and after logging in as bduffy was able to launch idlde w/o any license warnings.

============================================================
idlde libGL error on Sirius and Mirzam
============================================================

When I tested idlde by logging in as bduffy and launching idlde from Mirzam, idlde posted this warning message to it’s text window:

G’day!
% Compiled module: ADDPATH.
% Compiled module: AODEFDIR.
% Compiled module: P8.
% Compiled module: LDCOLPH.
libGL error: open DRM failed (Operation not permitted)
libGL error: reverting to (slow) indirect rendering
% Unsupported X Windows visual (class: PseudoColor, depth: 8).
Substituting default (class: TrueColor, Depth: 24).

NOTE: On algol, I get this message instead. It’s notable that the license wizard didn’t fail on algol…

G’day!
% Compiled module: ADDPATH.
% Compiled module: AODEFDIR.
% Compiled module: P8.
% Compiled module: LDCOLPH.
libGL warning: 3D driver claims to not support visual 0x4b
% Unsupported X Windows visual (class: PseudoColor, depth: 8).
Substituting default (class: TrueColor, Depth: 24).
% Compiled module: DOPHSBALL.

SOLUTION:

On mirzam I noticed that we didn’t see these errors running as root, and so Matt and I looked at /dev/dri/* and /dev/dri/card0 had perms of 600. When we chmod’d them to 666, idlde didn’t post the errors. So then the challenge was to find the right file that would change these perms at the right time. At first I thought that modding the line at the bottom of /etc/security/console.perms.d/50-default.perms would do it, but it didn’t work. We think it’s because the dri devices don’t exist at the time this file is applied at boot, but only after X is launched. The proper place/way to set the perms it to append these lines to /etc/X11/xorg.conf:

Section "DRI" Mode 0666 EndSection

Oddly, although applying this fix to Sirius seemed to work, there is no /dev/dri on Sirius, and running fglrxinfo segv’s, so something isn’t quite right…

.reset_session to take idl to the top

Published by jweisber on 2009-08-25

Joe S. discovered that the command .reset_session inside idl resets variables, pointers, etc , so that one does not have to entirely exit to recompile etc.

How fix “Firefox already in use” problem

Published by jweisber on 2009-07-23

Our shared home directories sometimes confuse Firefox, causing it to fail to launch while posting a dialog window saying that “Firefox is already in use”.

The only solution I’ve found that really works is somewhat drastic. It requires you to delete your ~/.mozilla/firefox directory, which has the effect of deleting your bookmarks.

To preserve your bookmarks, we must first export them to a temporary location, exit Firefox, delete your ~/.mozilla/firefox directory, relaunch Firefox, and then import your bookmarks from the temporary location.

Here’s the step by step solution:

1. To get Firefox to launch (in a somewhat hosed state) so that you can save your bookmarks, open a terminal window and issue these commands:

rm -f ~/.mozilla/firefox/*.default/lock
rm -f ~/.mozilla/firefox/*.default/.parentlock

2. Launch Firefox. It will be somewhat hosed, but you should be able to export your bookmarks by opening the Bookmarks window by selecting “Bookmarks->Organize Bookmarks” from the Browsers menu. Once the Bookmarks window appears, click on the “Import and Backup” menu and select “Backup”. A “Bookmarks backup” window will pop up prompting you for a location to save your bookmarks to. Just go with the default values, which should save a “Bookmarks <date>.json” file to your Desktop.

3. Exit Firefox

4. From a terminal window, issue this command:

rm -f ~/.mozilla/firefox/

5. Launch Firefox. At this point Firefox should behave normally and you can restore your bookmarks by opening the Bookmarks window and selecting “Bookmarks->Organize Bookmarks” from the Browsers menu. Once the Bookmarks window appears launched, click on the “Import and Backup” menu and select “Restore”. A dropdown list of of bookmark backup files should pop up, and you can select the most recent one to restore your bookmarks.

If IDL help browser will not come up . . .

Published by jweisber on 2009-06-19

Bruce figured out that the problem is that we share our home directory across all hosts, so idlhelp gets confused about which host it is on. (It does not keep track of this.)

So the fix is to get rid of the directory in your home directory that has the incorrect information:

in <homedirectory>/.idl/itt

type

rm -rf idlhelp

We think it is the .metadata/.connection file that causes the problem, but eliminating it does not do the job.

To find a file down a tree

Published by jweisber on 2009-06-18

find . -ls | grep -i <filename>

that means find all files from current dir and below, and get file date etc with ls and then find those matching <filename>. The ‘-i’ argument tells grep to ignore the case (uppercase/lowercase) of characters in the filename.

Carleton Astronet Admin Log Posts