by Jeff Hunter, Sr. Database Administrator
Overview
This month, I introduce the UNIX nohup utility. Most of the information included in this article are taken directly from the nohup man page.
As a DBA, you have more than likely been faced with the need to run a command in UNIX that you know will take a long time to complete. Most often, you will want to run the command and "exit" from the terminal. When the shell exits though, it sends its children a SIGHUP signal, which by default causes them to be killed. All stopped, running and background jobs will ignore SIGHUP and continue running if their invocation is preceded by the nohup or if the process programmatically has chosen to ignore SIGHUP.
The nohup utility invokes the named command with the arguments supplied. When the command is invoked, nohup arranges for the SIGHUP signal to be ignored by the process.
Syntax
/usr/bin/nohup command [ argument ...]
Processes run by /usr/bin/nohup are immune to SIGHUP (hangup) and SIGQUIT (quit) signals.
/usr/xpg4/bin/nohup command [ argument ...]
Processes run by /usr/xpg4/bin/nohup are immune to SIGHUP.
Other Notes
The nohup utility does not arrange to make processes immune to a SIGTERM (terminate) signal, so unless they arrange to be immune to SIGTERM or the shell makes them immune to SIGTERM, they will receive it.
If nohup.out is not writable in the current directory, output is redirected to $HOME/nohup.out. If a file is created, the file will have read and write permission "600". If the standard error is a terminal, it is redirected to the standard output, otherwise it is not redirected. The priority of the process run by nohup is not altered.
If you are running the Korn shell as your login shell, and have nohup'ed jobs running when you attempt to logout, you will be warned with the message
You have jobs running.
You will then need to logout a second time to actually logout; however, your background jobs will continue to run.
The C-shell has a built-in command nohup that provides immunity from SIGHUP, but does not redirect output to nohup.out. Commands executed with `&' are automatically immune to HUP signals while in the background.
nohup does not recognize command sequences. In the case of the following command
example$ nohup command1; command2
The nohup utility applies only to command1. The command
example$ nohup (command1; command2)
is syntactically incorrect.
Operands
The following operands are supported:
command
The name of a command that is to be invoked. If the command operand names any of the special shell_builtins(1) utilities, the results are undefined.
argument
Any string to be supplied as an argument when invoking the command operand.
Examples
It is frequently desirable to apply nohup to pipelines or lists of commands. This can be done only by placing pipe- lines and command lists in a single file, called a shell script. One can then issue:
example$ nohup sh file
and the nohup applies to everything in file. If the shell script file is to be executed often, then the need to type sh can be eliminated by giving file execute permission.
Add an ampersand and the contents of file are run in the background with interrupts also ignored (see sh(1)):
example$ nohup file &
Tuesday, June 30, 2009
Thursday, June 25, 2009
Difference RMAN and EXPDP
Question: I've been using data pump (the export utility) to backup my database, but I wonder if I should be using RMAN instead. They both do backups, so what's the advantages of each?
Answer: Yes, RMAN and export both backup tables, and they both support flashback database, but there are some important differences:
* Data Pump Export (expdp) - The export utility is a "logical" backup, usually done by specifying specific tables. If you fail to do a "consistent" export, or if you fail to include related tables and use RESTRICTED and CONSISTENT mode (with referential integrity constraints), you may not be able to recover properly. Also, export can only read "cooked" files, and database with raw files (e.g. RAC) cannot be copied with export. Hence, export is often used as a supplement to RMAN, usually for the restore of specific tables.
* Recovery manager (rman) - RMAN is designed for backup and recovery, a extension of the Enterprise Backup Utility (EBU). RMAN reads raw files, takes time-consistent backups. manages referential integrity.
Advantages & disadvantages of Export:
* Free, and easy to use
* Very slow, compared to RMAN (examines every data block)
* Easy restore of a specific table
* Does not require ARCHIVELOG mode
* Export will discover the SCN that is closest to the specified time. This SCN is used to enable flashback with export.
Advantages & disadvantages of RMAN:
* Supports raw devices
* RMAN has block-level media recovery
* Has a catalog for backup tracking and a report utility
* Fast - If you dedicate a backup device for each production disk, you can backup terabytes in the time it takes to backup any single disk.
* Does hot or cold backups
* Backups and restores can be done in parallel
* Allows incremental backups (block change tracking)
* Interfaces with media management systems (TMS)
* RMAN can be used to backup changed data even after a nologging operation, and you can move away from archivelog mode.
by Burleson Consulting - www.dba-oracle.com
Answer: Yes, RMAN and export both backup tables, and they both support flashback database, but there are some important differences:
* Data Pump Export (expdp) - The export utility is a "logical" backup, usually done by specifying specific tables. If you fail to do a "consistent" export, or if you fail to include related tables and use RESTRICTED and CONSISTENT mode (with referential integrity constraints), you may not be able to recover properly. Also, export can only read "cooked" files, and database with raw files (e.g. RAC) cannot be copied with export. Hence, export is often used as a supplement to RMAN, usually for the restore of specific tables.
* Recovery manager (rman) - RMAN is designed for backup and recovery, a extension of the Enterprise Backup Utility (EBU). RMAN reads raw files, takes time-consistent backups. manages referential integrity.
Advantages & disadvantages of Export:
* Free, and easy to use
* Very slow, compared to RMAN (examines every data block)
* Easy restore of a specific table
* Does not require ARCHIVELOG mode
* Export will discover the SCN that is closest to the specified time. This SCN is used to enable flashback with export.
Advantages & disadvantages of RMAN:
* Supports raw devices
* RMAN has block-level media recovery
* Has a catalog for backup tracking and a report utility
* Fast - If you dedicate a backup device for each production disk, you can backup terabytes in the time it takes to backup any single disk.
* Does hot or cold backups
* Backups and restores can be done in parallel
* Allows incremental backups (block change tracking)
* Interfaces with media management systems (TMS)
* RMAN can be used to backup changed data even after a nologging operation, and you can move away from archivelog mode.
by Burleson Consulting - www.dba-oracle.com
Sunday, January 25, 2009
Install .rpm Files in Debian and Ubuntu
Some time you might find some applications are having only .rpm files but you want a .deb package for your debian,Ubuntu and other debian derived ditributions.If you can’t find .deb debian package in any of the debian,ubuntu repositories or elsewhere, you can use the alien package converter to install the .rpm file.
Alien is a program that converts between the rpm, dpkg, stampede slp, and slackware tgz file formats. If you want to use a package from another distribution than the one you have installed on your system, you can use alien to convert it to your preferred package format and install it.
Despite the large version number, alien is still (and will probably always be) rather experimental software. It has been used by many people for many years, but there are still many bugs and limitations.
Alien should not be used to replace important system packages, like sysvinit, shared libraries, or other things that are essential for the functioning of your system. Many of these packages are set up differently by Debian and Red Hat, and packages from the different distributions cannot be used interchangably. In general, if you can’t uninstall the package without breaking your system, don’t try to replace it with an alien version.
Install alien in debian
#apt-get install alien
This will install all the required packages.Now you can start converting your .rpm files to .deb packages.
Available Options for alien
Convert the package.rpm into a package.deb
#alien -d package-name.rpm
Convert the package.rpm into a package.deb, and install the generated package.
#alien -i package-name.rpm
If you want to keeps alien from changing the version number use the following command
#alien -k rpm-package-file.rpm
Example
Suppose we have a avg antivirus avg71lms-r30-a0782.i386.rpm file
To convert .rpm to debian
#alien -k avg71lms-r30-a0782.i386.rpm
Now you should be having avg71lms-r30-a0782.i386.deb file
To install .deb file
#dpkg -i avg71lms-r30-a0782.i386.deb
If you don’t use -k option you should see avg71lms_r30-1_i386.deb file the difference is it will add 1
Install alien in Ubuntu
$sudo apt-get install alien
You can check the above section for available options
Example
Suppose we have a avg antivirus avg71lms-r30-a0782.i386.rpm file
To convert .rpm to debian
$sudo alien -k avg71lms-r30-a0782.i386.rpm
Now you should be having avg71lms-r30-a0782.i386.deb file
To install .deb file
$sudo dpkg -i avg71lms-r30-a0782.i386.deb
If you don’t use -k option you should see avg71lms_r30-1_i386.deb file the difference is it will add 1
Alien is a program that converts between the rpm, dpkg, stampede slp, and slackware tgz file formats. If you want to use a package from another distribution than the one you have installed on your system, you can use alien to convert it to your preferred package format and install it.
Despite the large version number, alien is still (and will probably always be) rather experimental software. It has been used by many people for many years, but there are still many bugs and limitations.
Alien should not be used to replace important system packages, like sysvinit, shared libraries, or other things that are essential for the functioning of your system. Many of these packages are set up differently by Debian and Red Hat, and packages from the different distributions cannot be used interchangably. In general, if you can’t uninstall the package without breaking your system, don’t try to replace it with an alien version.
Install alien in debian
#apt-get install alien
This will install all the required packages.Now you can start converting your .rpm files to .deb packages.
Available Options for alien
Convert the package.rpm into a package.deb
#alien -d package-name.rpm
Convert the package.rpm into a package.deb, and install the generated package.
#alien -i package-name.rpm
If you want to keeps alien from changing the version number use the following command
#alien -k rpm-package-file.rpm
Example
Suppose we have a avg antivirus avg71lms-r30-a0782.i386.rpm file
To convert .rpm to debian
#alien -k avg71lms-r30-a0782.i386.rpm
Now you should be having avg71lms-r30-a0782.i386.deb file
To install .deb file
#dpkg -i avg71lms-r30-a0782.i386.deb
If you don’t use -k option you should see avg71lms_r30-1_i386.deb file the difference is it will add 1
Install alien in Ubuntu
$sudo apt-get install alien
You can check the above section for available options
Example
Suppose we have a avg antivirus avg71lms-r30-a0782.i386.rpm file
To convert .rpm to debian
$sudo alien -k avg71lms-r30-a0782.i386.rpm
Now you should be having avg71lms-r30-a0782.i386.deb file
To install .deb file
$sudo dpkg -i avg71lms-r30-a0782.i386.deb
If you don’t use -k option you should see avg71lms_r30-1_i386.deb file the difference is it will add 1
Recover Data from a Damaged hard disk using dd_rhelp
dd_rhelp is a bash script that handles a very usefull program written in C which is called dd_rescue, it roughly act as the dd linux command with the caracteristic to NOT stop when it falls on read/write errors.
This makes dd_rescue the best tool for recovering hard drive having bad sectors.In short, it’ll use dd_rescue on your entire disc, but will try to gather the maximum valid data before trying for ages on badsectors. So if you leave dd_rhelp work for infinite time, it’ll have the same effect as a simple dd_rescue. But because you might not have this infinite time , dd_rhelp will jump over bad sectors and rescue valid data. In the long run, it’ll parse all your device with dd_rescue.
You can Ctrl-C it whenever you want, and rerun-it at will, it’ll resume it’s job as it depends on the log files dd_rescue creates.In addition, progress will be shown in a ASCII picture of your device beeing rescued.
Why use dd_rhelp and not dd_rescue ?
dd_rhelp uses dd_rescue to compute a recovery path through the device that will focus on valid data recovering. This recovery path will go through all the device, exactly as dd_rescue could do it on its own without any path. This means that dd_rhelp will save you time ONLY IF YOU INTEND TO CANCEL ITS JOB BEFORE THE END of a full recovery.
Because a considerable amount of time is taken to try to rescue badsectors. This amount of time can be mesured in days,
month, years, depending on your device capacity and its defectiveness. You might not want to spend this time knowing that 99 percent of this time will be taken to look at badsector and won’t lead to any more data recovering.
dd_rhelp shifts this useless waiting time to the end. Using dd_rescue strait throughout your device make your waiting time dependent on the badsector distribution.
Install dd_rhelp
First you need to download latest version from here
Preparing your system
#apt-get install make gcc g++ ddrescue
If you are using ubuntu linux use the following command
sudo apt-get install make gcc g++ ddrescue
Download dd_rhelp using the follwoing command
wget http://www.kalysto.org/pkg/dd_rhelp-0.0.6.tar.gz
Now you have dd_rhelp-0.0.6.tar.gz file
Extract this file using the following command
tar xzvf dd_rhelp-0.0.6.tar.gz
cd dd_rhelp-0.0.6
Run the following commands
# ./configure
# make
# make install
If you are using ubuntu linux use the following command
sudo ./configure
sudo make
sudo make install
This will complete the installation and this will install program under /usr/local/bin/dd_rhelp
dd_rhelp Syntax
dd_rhelp {filename|device} {output-file} [{info}]
or dd_rhelp --help
or dd_rhelp --version
Now we will see how to use ddrescue under damaged disk
If you have a damaged hard disk /dev/sda1 and you have an empty space hard disk /dev/sda2 Now if you want to copy data from /dev/sda1 to /dev/sda2 use the following command
#dd_rhelp /dev/sda1 /dev/sda2/backup.img
If you are using ubuntu linux use the following command
sudo dd_rhelp /dev/sda1 /dev/sda2/backup.img
Output looks like below
/usr/bin/tail: Warning: “+number” syntax is deprecated, please use “-n +number”
=== launched via ‘dd_rhelp’ at 0k, 0 >>> ===
dd_rescue: (info): ipos: 104384.0k, opos: 104384.0k, xferd: 104384.0k
errs: 0, errxfer: 0.0k, succxfer: 104384.0k
+curr.rate: 10178kB/s, avg.rate: 8532kB/s, avg.load: 31.3%
dd_rescue: (info): /dev/sda2 (104391.0k): EOF
Summary for /dev/sda2 -> backup.img:
dd_rescue: (info): ipos: 104391.0k, opos: 104391.0k, xferd: 104391.0k
errs: 0, errxfer: 0.0k, succxfer: 104391.0k
+curr.rate: 355kB/s, avg.rate: 8519kB/s, avg.load: 31.3%
/usr/bin/tail: Warning: “+number” syntax is deprecated, please use “-n +number”
This copies an image of /dev/sda1 to sda2
Now you need to check the backup image consistency this will check for is there any problems with this image.
#fsck -y /dev/sda2/backup.img
If you are using ubuntu linux use the following command
sudo fsck -y /dev/sda2/backup.img
After finishing this checking you need to mount your disk image in to your other hard disk
#mount /dev/sda2/backup.img /mnt/recoverydata
If you are using ubuntu linux use the following command
sudo mount /dev/sda2/backup.img /mnt/recoverydata
This will mount all the data from the backup.img under /mnt/recoverydata now you can try to access the data it should
work without any problem.
Restore image
If you want to restore this image use the following command
#dd_rhelp /dev/sda2/backup.img /dev/sda1
If you are using ubuntu linux use the following command
sudo dd_rhelp /dev/sda2/backup.img /dev/sda1
Copy Disk Image to remote machine using SSH
If you want to copy your disk image to remote machine over ssh you need to use the following command
#dd_rhelp /dev/sda1 - | ssh username@machineip ‘cat /datarecovery/backup.img’
If you are using ubuntu linux use the following command
sudo dd_rhelp /dev/sda1 - | ssh username@machineip ‘cat /datarecovery/backup.img’
This will be prompetd for password of the username you have menctioned in the above command after entering the password
dd_rhelp strats copying obviously it will take some time to copy over the network
This makes dd_rescue the best tool for recovering hard drive having bad sectors.In short, it’ll use dd_rescue on your entire disc, but will try to gather the maximum valid data before trying for ages on badsectors. So if you leave dd_rhelp work for infinite time, it’ll have the same effect as a simple dd_rescue. But because you might not have this infinite time , dd_rhelp will jump over bad sectors and rescue valid data. In the long run, it’ll parse all your device with dd_rescue.
You can Ctrl-C it whenever you want, and rerun-it at will, it’ll resume it’s job as it depends on the log files dd_rescue creates.In addition, progress will be shown in a ASCII picture of your device beeing rescued.
Why use dd_rhelp and not dd_rescue ?
dd_rhelp uses dd_rescue to compute a recovery path through the device that will focus on valid data recovering. This recovery path will go through all the device, exactly as dd_rescue could do it on its own without any path. This means that dd_rhelp will save you time ONLY IF YOU INTEND TO CANCEL ITS JOB BEFORE THE END of a full recovery.
Because a considerable amount of time is taken to try to rescue badsectors. This amount of time can be mesured in days,
month, years, depending on your device capacity and its defectiveness. You might not want to spend this time knowing that 99 percent of this time will be taken to look at badsector and won’t lead to any more data recovering.
dd_rhelp shifts this useless waiting time to the end. Using dd_rescue strait throughout your device make your waiting time dependent on the badsector distribution.
Install dd_rhelp
First you need to download latest version from here
Preparing your system
#apt-get install make gcc g++ ddrescue
If you are using ubuntu linux use the following command
sudo apt-get install make gcc g++ ddrescue
Download dd_rhelp using the follwoing command
wget http://www.kalysto.org/pkg/dd_rhelp-0.0.6.tar.gz
Now you have dd_rhelp-0.0.6.tar.gz file
Extract this file using the following command
tar xzvf dd_rhelp-0.0.6.tar.gz
cd dd_rhelp-0.0.6
Run the following commands
# ./configure
# make
# make install
If you are using ubuntu linux use the following command
sudo ./configure
sudo make
sudo make install
This will complete the installation and this will install program under /usr/local/bin/dd_rhelp
dd_rhelp Syntax
dd_rhelp {filename|device} {output-file} [{info}]
or dd_rhelp --help
or dd_rhelp --version
Now we will see how to use ddrescue under damaged disk
If you have a damaged hard disk /dev/sda1 and you have an empty space hard disk /dev/sda2 Now if you want to copy data from /dev/sda1 to /dev/sda2 use the following command
#dd_rhelp /dev/sda1 /dev/sda2/backup.img
If you are using ubuntu linux use the following command
sudo dd_rhelp /dev/sda1 /dev/sda2/backup.img
Output looks like below
/usr/bin/tail: Warning: “+number” syntax is deprecated, please use “-n +number”
=== launched via ‘dd_rhelp’ at 0k, 0 >>> ===
dd_rescue: (info): ipos: 104384.0k, opos: 104384.0k, xferd: 104384.0k
errs: 0, errxfer: 0.0k, succxfer: 104384.0k
+curr.rate: 10178kB/s, avg.rate: 8532kB/s, avg.load: 31.3%
dd_rescue: (info): /dev/sda2 (104391.0k): EOF
Summary for /dev/sda2 -> backup.img:
dd_rescue: (info): ipos: 104391.0k, opos: 104391.0k, xferd: 104391.0k
errs: 0, errxfer: 0.0k, succxfer: 104391.0k
+curr.rate: 355kB/s, avg.rate: 8519kB/s, avg.load: 31.3%
/usr/bin/tail: Warning: “+number” syntax is deprecated, please use “-n +number”
This copies an image of /dev/sda1 to sda2
Now you need to check the backup image consistency this will check for is there any problems with this image.
#fsck -y /dev/sda2/backup.img
If you are using ubuntu linux use the following command
sudo fsck -y /dev/sda2/backup.img
After finishing this checking you need to mount your disk image in to your other hard disk
#mount /dev/sda2/backup.img /mnt/recoverydata
If you are using ubuntu linux use the following command
sudo mount /dev/sda2/backup.img /mnt/recoverydata
This will mount all the data from the backup.img under /mnt/recoverydata now you can try to access the data it should
work without any problem.
Restore image
If you want to restore this image use the following command
#dd_rhelp /dev/sda2/backup.img /dev/sda1
If you are using ubuntu linux use the following command
sudo dd_rhelp /dev/sda2/backup.img /dev/sda1
Copy Disk Image to remote machine using SSH
If you want to copy your disk image to remote machine over ssh you need to use the following command
#dd_rhelp /dev/sda1 - | ssh username@machineip ‘cat /datarecovery/backup.img’
If you are using ubuntu linux use the following command
sudo dd_rhelp /dev/sda1 - | ssh username@machineip ‘cat /datarecovery/backup.img’
This will be prompetd for password of the username you have menctioned in the above command after entering the password
dd_rhelp strats copying obviously it will take some time to copy over the network
Subscribe to:
Posts (Atom)