1998-09-23 1. Partition and newfs the data disks - newfs -m 1 -i 8192 - keep only 1% for minfree and use 8192 bytes/inode - 9GB disks have a layout like: * First Sector Last * Partition Tag Flags Sector Count Sector Mount Directory 1 3 01 0 258552 258551 2 5 01 0 17682084 17682083 3 0 00 258552 7182 265733 6 4 00 265734 17416350 17682083 /export/home31 - where slice 1 is swap slice 3 is ~3 to 4MB for metadb replicas slice 6 is data space - two of the internal 4.2GB disks have a layout like: * First Sector Last * Partition Tag Flags Sector Count Sector Mount Directory 1 3 01 0 254880 254879 2 5 00 0 8380800 8380799 3 0 00 254880 8640 263519 4 0 00 263520 131760 395279 6 8 00 395280 7985520 8380799 /export/home02 - where slice 1 is swap slice 3 is ~3 to 4MB for metadb replicas slice 4 is 64MB reserved for UFS+Logging log space slice 6 is data space - the root disk is to be mirrored fully (incl. swap) to a disk on another controller; similarly with the /var/{log,spool,mail} disk - swap is a bit less than 128MB on each disk (minus the extra mirrored swap) 2. Set up the state databases for DiskSuite - need a minimum of 3, can have a maximum of 50 on the system - need (floor(50%) + 1) of all replicas available to keep the system alive or to allow it to boot. - put one replica per disk on each internal disk, plus two replicas in each of the external multipacks, one per disk. - this allows us to remove the external diskpacks, and still have state db quorum of > 50% *even if one system drive dies* - also spreads out the load over 4 controllers - rationale gained from SDS 4.1 Users' Guide page 4 - create the metadb replicas (the -f flag is only used on the initial create) metadb -a -f c0t0d0s3 c0t1d0s3 c0t2d0s3 c0t3d0s3 c2t0d0s3 c2t1d0s3 c3t1d0s3 c3t2d0s3 c4t1d0s3 c4t2d0s3 1998-09-24 3. Set up mirrors of "rootvg" type filesystems. a. do the 'umount'-able ones first (/var/mail, /var/log, /var/spool) - set up a one-way mirror of the data-full slice, then attach the second sub-mirror for a resync (/var/mail) metainit -f d51 1 1 c0t1d0s6 metainit d52 1 1 c2t1d0s6 metainit d50 -m d51 - umount the master (data-full) partition - edit /etc/vfstab to reflect the new metadevice: #/dev/dsk/c0t1d0s6 /dev/rdsk/c0t1d0s6 /var/mail ufs 4 yes - /dev/md/dsk/d50 /dev/md/rdsk/d50 /var/mail ufs 4 yes - - attach the second submirror: metattach d50 d52 - similar for the other partitions. d40 (d41, d42) for /var/log d50 (d51, d52) for /var/mail d60 (d61, d62) for /var/spool d70 (d71, d72) for /usr/vice/cache b. do the ones that can't be 'umount'-ed (/var, /opt) - set up a one-way mirror of the data-full slice, edit /etc/vfstab to refer to the metadevice, reboot, then use metattach to attach the second sub-mirror (/var) - d30 (d31, d32) metainit -f d31 1 1 c0t0d0s5 metainit d32 1 1 c2t0d0s5 metainit d30 -m d31 - edit /etc/vfstab and reboot - attach the second submirror: metattach d30 d32 - similar for /opt - d20 (d21, d22) c. do mirrors of the swap spaces on the two system drives - treat them like 'umount'-able partitions d1 (d3, d4) - c[02]t0d0s1 d2 (d5, d6) - c[02]t1d0s1 metainit -f d3 1 1 c0t0d0s1 metainit d4 1 1 c2t0d0s1 metainit d1 -m d3 - similar for d2 (d5, d6) - edit /etc/vfstab - try just onlining the new swap md's ** if it fails, try a reboot ** (should work (on this machine anyway) since swap is currently 0% used) metattach d1 d4 metattach d2 d6 d. mirror of / - d10 (d11, d12) metainit -f d11 1 1 c0t0d0s0 metainit d12 1 1 c2t0d0s0 metainit d10 -m d11 metaroot d10 - metaroot does the /etc/vfstab edit for you ( to make sure it gets done right, I guess, since there is nothing different about this edit vs the earlier ones.) lockfs -fa (*NOTE* the SDS 4.1 book says use "lockfs -fa" but the "-f" flag is not reported in the lockfs(1m) manpage) reboot metattach d10 d12 ls -l /dev/rdsk/c2t0d0s0 - to record the details on the alternate root slice /dev/rdsk/c2t0d0s0 -> ../../devices/pci@4,4000/scsi@2/sd@0,0:a,raw - supposedly, you need that long-winded string to boot the system off of the alternate root disk, but according to BobB and Antoine, you can use the prom-level disk alias (like "boot disk4" or something) - to be able to use the prom alias for the alt root disk, one should determine it, and record it (on paper) ** ** ** FIND THE PROM-LEVEL DISK ALIAS FOR c2t0d0s0 ** ** 1998-09-25 I've done all that I can find to make it boot from the c2t0d0s0 disk, but no dice. - at the PROM level, I did nvalias altroot /pci@4,4000/scsi@2/sd@0,0:a nvstore reset-all boot altroot - get a failure of: Drive not ready Can't read disk label. Can't read disk label package Boot device net ... - doing "boot /pci@4,4000/scsi@2/sd@0,0:a" fails in the same way. - moving the c2t0d0s0 disk to the c0t0d0s0 position and doing "boot disk" or "boot" *works fine* 1998-10-13 I've found the secret to making an nvram alias that is bootable: Instead of using /pci@4,4000/scsi@2/sd@0,0:a you need to use /pci@4,4000/scsi@2/disk@0,0:a ^^ ^^^^ Now, doing at the PROM: nvalias altroot /pci@4,4000/scsi@2/sd@0,0:a you can do "boot altroot" and have it work. After doing recovery work, and booting from the normal boot disk again, the mirrors will show that they need maintenance. After doing a test, I needed to reenable the submirrors with: metareplace -e d10 c0t0d0s0 Doing "metastat | grep replace" will give a hint as to which submirrors need to have manual intervention. 4. Backups - To backup / do the following: * Note: DO NOT use lockfs on / as that causes the system to hang metaoffline d10 d12 ufsdump 0uf /dev/rmt/0cn /dev/md/rdsk/d12 metaonline d10 d12 # automatically re-sync's - To backup other partitions, do: /usr/sbin/lockfs -w metaoffline d30 d32 /usr/sbin/lockfs -u ufsdump 0uf /dev/rmt/0cn /dev/md/rdsk/d32 metaonline d30 d32 5. Filesystem tweaks and installations - symlinks: /usr/X11 -> ./openwin /usr/lib/X11 -> ../openwin/lib - replacement programs: /usr/lib/sendmail ( gets 8.8.8 ) /usr/lib/mail.local ( from 8.8.8 ) 6. Set up stripe for OpenBSD anoncvs metainit d7 1 3 c4t2d0s0 c4t4d0s0 c4t8d0s0 -i 16k metainit d8 1 3 c4t2d0s6 c4t4d0s6 c4t8d0s6 -i 16k Sets up two stripes, of three slices each, with the interlace size set to 16k (the default). The first of these has slices of ~1.5 GB wide, and the second has slices ~0.5 GB wide. The first will be for the OpenBSD anonCVS tree. FS parameters are as follows: newfs /dev/md/rdsk/d7 (minfree = 10%, default 4k/inode) The second will potentially be used for Oracle logs, and is set up as follows: newfs -m 1 -i 8192 /dev/md/rdsk/d8 ================================================================================= 1999-01-08 E250 setup 1. Set up the state databases for DiskSuite - need a minimum of 3, can have a maximum of 50 on the system - need (floor(50%) + 1) of all replicas available to keep the system alive or to allow it to boot. - put two replicas per disk on each internal disk, plus two replicas in each of the external multipacks, one per disk. - this allows us to remove the external diskpacks, and still have state db quorum of > 50% *even if one system drive dies* - also spreads out the load over at least 2 controllers - rationale gained from SDS 4.1 Users' Guide page 4 - create the metadb replicas (the -f flag is only used on the initial create) metadb -a -c 2 -f c0t0d0s3 c0t8d0s3 c0t9d0s3 metadb -a c1t1d0s3 c1t2d0s3 2. Set up mirrors of "rootvg" type filesystems. a. do the 'umount'-able ones first (/usr/vice/cache) - set up a one-way mirror of the data-full slice, then attach the second sub-mirror for a resync (/usr/vice/cache) metainit -f d41 1 1 c0t0d0s7 metainit d42 1 1 c0t8d0s7 metainit d40 -m d41 - umount the master (data-full) partition - edit /etc/vfstab to reflect the new metadevice: #/dev/dsk/c0t0d0s7 /dev/rdsk/c0t0d0s7 /usr/vice/cache ufs 2 yes - /dev/md/dsk/d40 /dev/md/rdsk/d40 /usr/vice/cache ufs 2 yes - - attach the second submirror: metattach d40 d42 b. do the ones that can't be 'umount'-ed (/var, /opt) - set up a one-way mirror of the data-full slice, edit /etc/vfstab to refer to the metadevice, reboot, then use metattach to attach the second sub-mirror (/var) - d20 (d21, d22) metainit -f d21 1 1 c0t0d0s4 metainit d22 1 1 c0t8d0s4 metainit d20 -m d21 - edit /etc/vfstab and reboot - attach the second submirror: metattach d20 d22 - similar for /opt - d30 (d31, d32) c. do mirrors of the swap spaces on the system drive - treat them like 'umount'-able partitions d1 (d3, d4) - c[02]t0d0s1 metainit -f d3 1 1 c0t0d0s1 metainit d4 1 1 c0t8d0s1 metainit d1 -m d3 - edit /etc/vfstab - try just onlining the new swap md's ** if it fails, try a reboot ** (should work (on this machine anyway) since swap is currently 0% used) metattach d1 d4 d. mirror of / - d10 (d11, d12) metainit -f d11 1 1 c0t0d0s0 metainit d12 1 1 c0t8d0s0 metainit d10 -m d11 metaroot d10 - metaroot does the /etc/vfstab edit for you ( to make sure it gets done right, I guess, since there is nothing different about this edit vs the earlier ones.) lockfs -fa (*NOTE* the SDS 4.1 book says use "lockfs -fa" but the "-f" flag is not reported in the lockfs(1m) manpage) reboot metattach d10 d12 - record the details on the alternate root slice - on the E250, you only need to do "boot disk1" to get the alternate root