Skip Headers

Oracle® Database Backup and Recovery Advanced User' s Guide
10g Release 1 (10.1)

Part Number B10734-01
Go to Documentation Home
Home
Go to Book List
Book List
Go
to Table of Contents
Contents

Index
Go to Master Index
Master Index
Go to Feedback page
Feedback

Go to previous page
Previous
Go to next page
< font size="-2">Next
View PDF

20
Troubleshooting Use r-Managed Media Recovery

This chapter describes how to troubl eshoot user-managed media recovery, and includes the following topics:

About User-Managed Media Recovery Problems

Table 20-1 describes potential problems that can occur during media recovery.

< a name="1006187">
Table 20-1 Media Recovery Problems  

The symptoms of media recovery problems are usually external or internal errors signaled during recovery. For example, an external error indicates that a redo block or a data block has failed checksum verification checks. Internal errors can be caused by either bugs in the database or errors arising from the underlying operating system and hardware.

If media recovery encounters a problem while recovering a database backup, whether it is a stu ck recovery problem or a problem during redo application, the database always stops and leaves the datafiles undergoing recovery in a consistent state, that is, at a consistent SCN preceding the failure. You can then do one of the following:

  • Open the database read-only to investigate the problem.
  • Open the database with the RESETLOGS option, as long as the requirements for opening RESE TLOGS have been met. Note that the RESETLOGS restrictions apply to opening the standby database as well, because a standby database is updated by a form of media recovery.

In general, opening the da tabase read-only or opening with the RESETLOGS option require all online datafiles to be recovered to the same SCN. If t his requirement is not met, then the database may signal ORA-1113 or other errors when you attempt to open. Some common causes of ORA-1113 are described in Table 20-1.

The basic methodology for responding to media recovery problems occurs in the following phases:

  1. Try to identify the cause of the problem. Run a trial recovery if needed.
  2. If the problem is related to missing redo logs or you suspect there is a redo log, memory, or data block corruption, then try to resolve it using the methods described in T able 20-2.
  3. If you cannot resolve the problem using the methods described in Table 20-2, then do one of the following:
    • Open the database with the RESETLOGS option if you are recovering a whole database back up. If you have performed serial media recovery, then the database contains all the changes up to but not including the changes at th e SCN where the corruption occurred. No changes from this SCN onward are in the recovered part of the database. If you have restored online backups, then opening RESETLOGS succeeds only if you have recovered through all the ALTER ...< /code> END BACKUP operations in the redo stream.
    • P roceed with recovery by allowing media recovery to corrupt data blocks. After media recovery completes, try performing block media re covery using RMAN.
    • Call Oracle Support Services as a last resort.

Problem Description

Missing or misnamed archived log

Recovery stops because the database cannot find the archived log recorded in the control file.

When you attempt to open the database, error ORA-1113 indicates that a datafile needs media recovery

This error commonly occurs because:

  • You are performing incomplete recovery but failed to restore all needed datafile backups.
  • Incomplete recovery stopped before datafiles reached a consistent SCN.
  • You are recovering datafiles from an online backup, but not enough redo was applied to make the datafiles consistent.
  • You are performing recovery with a backup control file, and did not specify the location of a ne eded online redo log.
  • A datafile is undergoing media recovery when you attempt to open the database.
  • Datafiles needing recovery were not brought online befo re executing RECOVER DATABASE, and so were not recovered.

Redo record problems

Two possible cases are as follows:

  • Recovery stops because of failed consistency checks, a problem called stuck recovery. Stuck re covery can occur when an underlying operating system or storage system loses a write issued by the database during normal operation.< /li>
  • The database signals an internal error when applying the redo. This problem ca n be caused by an Oracle bug. If checksums are not being used, it can also be caused by corruptions to the redo or data blocks.
  • < /ul>

Corrupted archi ved logs

Logs may be corrupted while they are stored on or copied b etween storage systems. If DB_BLOCK_CHECKSUM is enabled, then the database usually signals checksum errors. If checksumm ing is not on, then log corruption may appear as a problem with redo.

Archived logs with incompatible parallel redo format

If you enable the parallel redo feature, then the database generates redo logs in a new forma t. Prior releases of Oracle are unable to apply parallel redo logs. However, releases prior to Oracle9i Relea se 2 (9.2) can detect the parallel redo format and indicate the inconsistency with the following error message: External error 00303, 00000, "cannot process Parallel Red o".

See Also: Oracle Database Performance Tuning Guide to learn about the parallel redo feature

Corrupted data blocks

A datafile backup may have contained a corrupted data block, or the data block may become corrupted either during recovery or when it was copied to the backup. If checksums are being used, then the database signals a checksum error. Otherwise, the problem may also appear as a redo corruption.

Random problems

Memory corruptions and other transient problems can occur during recovery.

See Also:

"Performing Block Media Recovery with RMAN" to learn about block media recovery

Investigating the Media Recovery Problem: Phase 1

If media recovery encounters a problem, then obtain as much information as possible after recovery halts. You do not want to waste time fixing the wrong problem, which may in fact make mat ters worse.

The goal of this initial investigation is to determine whether the problem is c aused by incorrect setup, corrupted redo logs, corrupted data blocks, memory corruption, or other problems. If you see a checksum err or on a data block, then the data block is corrupted. If you see a checksum error on a redo log block, then the redo log is corrupted .

Sometimes the cause of a recovery problem can be difficult to determine. Nevertheless, th e methods in this chapter allow you to quickly recover a database even when you do not completely understand the cause of the problem .

To investigate media recovery problems:

  1. Examine the alert.log to see whether the error messa ges give general information about the nature of the problem. For example, does the alert_SID.log indicate any checksum failures? Does the alert_SID.log indicate that med ia recovery may have to corrupt data blocks in order to continue?
  2. Check the trace file generated by the Oracle process during recovery. It may contain additional error information.

Trying to Fix the Recovery Problem Without Corrupting Blocks: Phase 2

D epending on the type of media recovery problem you suspect, you have different solutions at your disposal. You can try one or a combi nation of the methods described in Table 20-2. Note that these methods are fairly safe: in al most all cases, they should not cause any damage to the database.

< a name="1006244"> Table 20-2 Media Recovery Solutions  
If you suspect . . . Then . . .

Missing/misnamed archived logs

Determine whether you entered the correct filename. If you did, then check to se e whether the log is missing from the operating system. If it is missing, and you have a backup, then restore the backup and apply th e log. If you do not have a backup, then if possible perform incomplete recovery up to the point of the missing log.

ORA-1113 for< /code> ALTER DATABASE OPEN

R eview the causes of this error in Table 20-1. Make sure that all read/write datafiles requiri ng recovery are online. If you use a backup control file for recovery, then the control file and datafiles must be at a consistent SC N for the database to be opened. If you do not have the necessary redo, then you must re-create the control file.

Corrupt archived logs

The log is corrupted if the checksum verification on the log redo block fails. I f DB_BLOCK_CHECKSUM is not enabled either during the recovery session or when the database generated the redo, then reco very problems may be caused by corrupted logs. If the log is corrupt and an alternate copy of the corrupt log is available, then try to apply it and see whether this tactic fixes the problem.

The DB_BLOCK_CHECKSUM initialization parameter determines whether checksums are computed for redo log and data blocks.

Archived logs with incompatible parallel redo form at

If you are running an Oracle release prior to Oracle9i Release 2, and if you are attempting to apply redo logs created with the parallel redo format, then you must do the fo llowing steps:

  1. Upgrade the database to a later release.
  2. Perform media recovery.
  3. Shut down the database consistently and back up the database.
  4. Downgrade the database to the original release.

See Also: Oracle Database Performance Tuning Guid e to learn about the parallel redo feature

Memory corruption or transient problems

You may be able to fix the problem by shutting down the database and restarting recovery. The databse should be left in a co nsistent state if the second attempt also fails.

Corrupt data blocks

Restore and re cover the datafile again with user-managed methods, or restore and recover individual data blocks with the RMAN BLOCKRECOVER command. This tactic may fix the problem.

A data block is corrupted if the checksum ver ification on the block fails. If DB_BLOCK_CHECKING is disabled, a corrupted data block problem may appear as a redo prob lem. If you must proceed with recovery, then you may want to corrupt the block now and continue recovery, and use RMAN to perform blo ck media recovery later.

If you cannot fix the problem with the m ethods described in Table 20-2, then there may be no easy way to fix the problem without losi ng data. You have these options:

  • Open the database with the RESETLOGS option (for whole database recovery). This solution discards all changes after the point where the redo problem occurred, but guarantees a logically consistent database.
  • Allow media recovery to corrupt one or more data blocks and proceed with media recovery. This option will only succeed if the alert_SID.log indicates that recovery can continue if it is allowed to corrupt a data block, which should be the case for most recovery problems. This option is best if it is important to bring up the database quickly and recover all changes. If you are contemplating this option as a last resort, then proceed to "Deciding Whether to Allow Recove ry to Corrupt Blocks: Phase 3".

    See Also:

    "Performing Block Media Recovery with RMAN" to learn how to perform block media recovery with the BLOCKRECOVER command

Deciding Whether to Allow Recovery to Corrupt Block s: Phase 3

When media recovery encounters a problem, the a lert_SID.log may indicate that recovery can continue if it is allowed to corrupt the data b lock causing the problem. The alert_SID.log always contains information about the blo ck: its block type, block address, the tablespace it belongs to, and so forth. For blocks containing user data, the alert log may als o report the data object number.

In this case, the database can proceed with recovery if it is allowed to mark the problem block as corrupt. Nevertheless, this response is not always advisable. For example, if the block is a n important block in the SYSTEM tablespace, marking the block as corrupt can eventually prevent you from opening the rec overed database. Another consideration is whether the recovery problem is isolated. If this problem is followed immediately by many o ther problems in the redo stream, then you may want to open the database with the RESETLOGS option.

For a block containing user data, you can usually query the database to find out which object or table owns thi s block. If the database is not open, then you should be able to open the database read-only, even if you are recovering a whole data base backup. The following example cancels recovery and opens read-only:

CANCEL
ALTER DATABASE OPEN READ ONLY;

Assume that the data ob ject number reported in the alert_SID.log is 8031. You can determine the owner, object name, and object type by issuing this query:

SELECT OWNER, OBJECT_NAME, SUBO
BJECT_NAME, OBJECT_TYPE
FROM DBA_OBJECTS
WHERE DATA_OBJECT_ID = 8031;
<
/a>

To determine whether a recovery problem is isolated, you can run a diagnostic trial recovery, which scans the redo stream for problems but does not actually make any changes to th e recovered database. If a trial recovery discovers any recovery problems, it reports them in the alert_SID.log. You can use the RECOVER ... TEST statement to invoke trial recover y.

See Also:

"Performing Trial Recovery"

< /td>

After you have done these investigations, you can follow the guidelines in Table 20-3 to decide whether to allow recovery to corrupt blocks.

< /a>
Table 20-3 Gu idelines for Allowing Recovery to Permit Corruption  
If the problem is . . . and the block is . . . Then . . .

not isolated

n/a

You should probably open the database wit h the RESETLOGS option. This response is important for stuck recovery problems, because stuck recovery can be caused by the operating system or a storage system losing writes. If an operating system or storage system suddenly fails, it can cause stuck r ecovery problems on several blocks.

isolated

in the SYSTEM tablespace< /p>

Do not corrupt the block, because it may eventually prevent you fro m opening the database. However, sometimes data in the SYSTEM tablespace is unimportant. If you must corrupt a SYS TEM block and recover all changes, contact Oracle Support.

isolated

index da ta

Consider corrupting index blocks because the index can be rebuil t later after the database has been recovered.

isolated

user data

Decide based on the importance of the data. If you continue with datafile recovery a nd corrupt a block, you lose data in the block. However, you can use RMAN to perform block media recovery later after datafile recove ry completes. If you open RESETLOGS, then the database is consistent but loses any changes made after the point where re covery was stopped.

isolated

rollback or undo data

Consider corrupting the rollback or undo block because it does not harm the database if the transac tions that generated the undo are never rolled back. However, if those transactions are rolled back, then corrupting the undo block c an cause problems. If you are unsure, then call Oracle Support.

See Also:

"Performing Trial Recovery" to learn how to perform trial recovery, and "Allowing Recovery to Corrupt Blocks: Phase 4" if you decide to corru pt blocks

Allowing Recovery to Corrupt Blocks: Phase 4

If you decide to allow recovery to proceed in spite of block corruptions, then run the RECOVER command with the ALLOW n CORRUPTION clause, where n is the numb er of allowable corrupt blocks.

To allow recovery to corrupt blocks:

  1. Ensure that all normal recovery precon ditions are met. For example, if the database is open, then take tablespaces offline before attempting recovery.
  2. Run the RECOVER command, allowing a single corruption, repeating as necessary for each corruption to be made. The following statements shows a valid example:
    RECOVER DATAB
    ASE ALLOW 1 CORRUPTION
    

Performing Trial Recovery

When problems such as stuck recovery occur, you have a difficult choice. If the block is relatively unimportant, and if the p roblem is isolated, then it is better to corrupt the block. But if the problem is not isolated, then it may be better to open the dat abase with the RESETLOGS option.

Because of this situation, the Oracle databas e supports trial recovery. A trial recovery applies redo in a way similar to normal media recovery, but it never writes its changes t o disk and it always rolls back its changes. Trial recovery occurs only in memory.

< table class="NoteAlso" border="0" width="80%" cellpadding="0" cellspacing="0" dir="ltr" summary="This is a layout table to format a n ote" title="This is a layout table to format a note"> See Also:

"Allowing Recovery to Corrupt Blocks: Phase 4"

How Trial Recovery Work s

By default, if a trial recovery encounters a stuck recovery or sim ilar problem, then it always marks the data block as corrupt in memory when this action can allow recovery to proceed. The database w rites errors generated during trial recovery to alert files. These errors are clearly marked as test run errors.

Like normal media recovery, trial recovery can prompt you for archived log filenames and ask you to apply them. Trial recovery ends when:

  • The database runs out of the maximu m number of buffers in memory that trial recovery is permitted to use
  • An unrec overable error is signaled, that is, an error that cannot be resolved by corrupting a data block
  • You cancel or interrupt the recovery session
  • The next redo record in the redo stream changes the control file
  • All requested redo has been applied

When trial recovery ends, the database removes all effects of the test run f rom the system--except the possible error messages in the alert files. If the instance fails during trial recovery, then the database removes all effects of trial recovery from the system because trial recovery never writes changes to disk.

Trial recovery lets you foresee what problems might occur if you were to continue with normal recovery. For problems caused by ongoing memory corruption, trial recovery and normal recovery can encounter different errors.

< !--TOC=h2-"1006462"-->

Executing the RECOVER ... TEST Statem ent

You can use the TEST option for any RECOVER command. For example, you can start SQL*Plus and then issue any of the following commands:

RECOVER DATABASE TEST
RECOVER DATABASE USING BACKUP CONTROLFILE UNTIL CANCEL TEST
R
ECOVER TABLESPACE users TEST
RECOVER DATABASE UNTIL CANCEL TEST

By default, trial recovery always attempts to corrupt blocks in memory if this action allows trial recovery to p roceed. In other words, trial recovery by default can corrupt an unlimited number of data blocks. You can specify the ALLOW n CORRUPTION clause on the RECOVER ... TEST statement t o limit the number of data blocks trial recovery can corrupt in memory.

A trial recovery co mmand is usable in any scenario in which a normal recovery command is usable. Nevertheless, you should only need to run trial recover y when recovery runs into problems.