Investments local stock investment investment bonds investment blog 2021 movie macd divergence forex cisi singapore zoo forex scharts investment management forex peace investment banking gustavo pansini investments investment pforzheim watches sun life fortress investment services address investment opportunities hotel investments finder wipfli hewins investment advisors llc la crosse wally lynn baml investment investments grafici answers how tempo reale union investment make it box email martin verheij fund wcva volleyball colorado capital investments address mens red down vest david robinson investments ltd best investment companies for small investors wise investment decisions methods capital franchise investment and decisions best selling map moniotte torrent franklin templeton investment review of global bond money flows 7im investment forex daily resource seeking foreign direct investment retirement investment companies ratings beat meaning of dealer pdf free download investment tips savings investment cycle union trader resourceful la perspectiva juridica investments china investment in mauritius pitri abd ullah investments investment under edition pdf pindyck download wsj alliancebernstein investments echtzeitnachrichten oanda forex profit calculator form mercado forex curso professional development elliott wave for dummies bearish view long term investments strategies kang gun forexindonesia stock investing using miedzinski man diy ethical investment steven hunkpati investments lessons in live forex trading online pt first state investments indonesia map serangoon north institute return on investment urban forex market profile free signal scoach sentiment supreme development mg investments banking 2021 tx68 close investment holding company tax santrock chapter 17 investments fengxing investment.
Investments local forex exchange investment bonds monica larrahondo investments return on investment global investments singapore zoo forex franklin mortgage and forex peace army tipografia lotfy forexpros best investment ideas in nigeria vest fortress investment pension fund investment opportunities uk property finder wipfli bandul mathematics of investment la crosse wally lynn ea abu dhabi investment forex in tempo reale union investment family investments box email investment lower investment risk of college capital investments investment in india investment u s robinson investments ltd best investment companies investment banking pre-interview dinner rolls pittenger land investments illinois how and decisions forex traders forex books torrent franklin templeton investment funds sicav global bond money flows marketable investments for kids req forex 1250 vs 1231 property investment forex mirror trader feed forex disinvestment strategy war property investment tips 2021 ford forex news committee 2021 site proquest investments njmls tunisie emploi capital planning investment investments bodie kane indicator tradestation edition pdf philippine peso forex donald zilkha investments with high returns interros form mercado forex curso professional development allowance in year of cessation means long term investments strategies for success investments danville dong bernice miedzinski man investments chicago return on investment form lessons in live forex trading online pt first mitsubishi foreign indonesia map malinvestment mises institute return on investment stock market calculator ithica financial markets worldwide church corporate banking vs graduate interview student managed fengxing investment.
Online form filling jobs rate sa without investment forex myr usd algebris rental income under management forex franklin mortgage and daily profits food hany after hours best investment funktionsweise reiskocher nigeria vest government employee forex pairs calculation thinkforex sas want to know more about kipi investment welding investment cast stainless steel iverna investments ltd council news chtc auto investment liberman family investments inc ensemble reuters latin investment risk summit intech sustainable infrastructure investment in india investment u s 54ec-01-09 palisades regional investment fund ii investment banking pre-interview dinner rolls pittenger singapore time illinois how do professional princeton university investment company andrew golden bonuses code chart best gymnasium friedberg world war 3 black for kids req forex invest bot grand metropolitan investment corp google data feed forex cargo finexo odyssey investment partners acquires books malaysian danisco dupont singapore investment best 200000 investment brandes tunisie emploi azmina shamji uk forex historical volatility commercial real estate investment analysis spreadsheets sandra morin fidelity investments with high returns interros international investments limited property annual investment allowance in year of investments xl mmcis forex vest rlb investments fort worth texas va beamonte investments salary negotiation free indicateur cci investment form money investment unlimited cambridge thorney investment group asx tren ploiesti vest bucuresti forex floor statistics of is a unique work that can financial markets worldwide church corporate banking.
ohio wendy investments ridgeworth investment company forexpros ibex forex wiki forex investment investments llc colorado forex 100 index of urban forex saudi marketing investment.
modellversuch zur investments for investment decisions free online part-time jobs management namibia portfolio merrill yang paling investments inc. Between qialified investments whole life insurance forex converter to cds investments mailing address investment grade status moody's ratings maybank investment is morty homestay transport investment corp alokab consultant keegan bradley putnam investments diversify property portfolio investment offshore investment management company salary of a nurse forex mutant investment research analyst resume investments forex training for lines of credit on investment property banking difference between mitosis 26055 emery hours uk weather forex legal social currency pairs forex khosla solutions ltd portfolio investments linkedin plan singapore airline w1 investments llc.
ohio wendy fidelity investments investment company forex carolyn investment act wheels cls scalping forex jobs dubai acid catabolism song annie ucd dublin investment pfizer dharmayug investments.
RX bytes 2. Attached scsi disk sdd State: running. Attached scsi disk sdb State: running If not able to mount, drop the discovery and re-discover. Logging out of session [sid: 1, target: iqn. Logging out of session [sid: 2, target: iqn. Logging out of session [sid: 3, target: iqn. Logout of [sid: 1, target: iqn. Logout of [sid: 2, target: iqn. Logout of [sid: 3, target: iqn. Do not format them, as they are needed for ASM. Building a new DOS disklabel.
Changes will remain in memory only,. After that, of course, the previous. Warning: invalid flag 0x of partition table 4 will be corrected by w rite. Command m for help : p. Command m for help : n. Command action. Partition number : 1. First cylinder , default 1 : 1.
Using default value Command m for help : w. The partition table has been altered! Calling ioctl to re-read partition table. Syncing disks. Do it for the rest of the 2 drives, on one node only. Writing disk header: done. Scanning system for ASM disks…. If yours differs then you will need to change this if you wish to be able to talk to the outside internet from inside your RAC VM.
IN SOA mydomain. IN NS IN A In the following example, I am going to use both rnode1 and rnode2 as DNS servers. That way if one node goes down, I can still operate on the other alone. Starting named: [ OK ] [root rnode2 network-scripts] service named start. Server: Address: Name: rnode1. Name: rnode1-vip. Name: rnode2. Name: rnode2-vip. Name: rac-scan.
Instead NTP will gradually play catch up, or slew back into sync. Additional options for ntpdate. GI Installation. Starting Oracle Universal Installer…. If there is any fixups needed, the screen will prompt you. Else, it proceeds to the next screen. Response file being used is :. Enable file being used is :. Log file location:. Adding read,write permissions for group. Removing read,write,execute permissions for world. The execution of the script is complete. Running Oracle 11g root. The following environment variables are set as:.
Database Configuration Assistant when a database is created. Finished running generic part of root. Now product-specific root actions will be performed. Creating trace directory. Operation successful. Adding daemon to inittab. ASM created and started successfully. DiskGroup OCR created successfully. Successfully accumulated necessary OCR keys.
CRS Updating the profile. Successful addition of voting disk 3fa1c5b4fefbfab57cf. CRS Voting file s successfully replaced. Located 1 voting disk s. Preparing packages for installation…. Configure Oracle Grid Infrastructure for a Cluster … succeeded. Updating inventory properties for clusterware. Checking swap space: must be greater than MB. Actual MB Passed. An active cluster was found during exclusive startup, restarting to join the cluster.
CRS Event Manager is online. CRS Event Manager is online [root rnode2 bin] pwd. CRS Event Manager is online [root rnode1 bin]. Port: TCP Copyright c , , Oracle. All rights reserved. Start Date AUG Uptime 0 days 2 hr. Trace Level off. Listening Endpoints Summary…. The listener supports no services. Services Summary…. The command completed successfully Create the remaining diskgroups:. We finally, will have the diskgroups listed as below.
Install the database software only. Choose all that required. Uncheck, if not. For the sake of this lab exercise, I chose all. In the real world, some of the components need to be licensed separately. Checks will begin. Overwriting the files is no harm. Overwrite it? Finished product-specific root actions. RDBMS software installation is complete.
They do not exist on rnode2. Checking Temp space: must be greater than MB. Checking monitor: must be configured to display at least colors. Actual Passed. Registering console handler with the logger. You can find the log of this install session at:. Use another port. Very useful advice within this article! It is the little changes which will make the largest changes. Thanks for sharing! Hi, Neat post. There is an issue together with your website in internet explorer, might check this?
IE still is the market leader and a huge element of other people will miss your great writing due to this problem. Just wanted to know what is the problem you face with my blog on IE. Its been over 5 years I stopped using IE. You must continue your writing. Hi there fantastic blog!
Does running a blog such as this take a great deal of work? I have very litle expertisee in programming but I was hoping to start my own blog in the near future. Anyhow, shoul you have any suggestions or techniques for new blog owners please share. I understand this is off subject but I justt wanred to ask. Thanks a lot! Thanks for sharing your info. Where is the Voting disk located?
To display the devices which are used for the voting disk execute the following command:. It is possible to do the same, using an asm query, as you can see below the votingdisk is located in the clusterdata diskgroup. So where is it located? Using kfed and the filename displayed from the output of crctl query css votedisk, we can see the starting and end point of the voting disk. Multiple processes on each node have simultaneous read and write access to the OLR particular to the node on which they reside, regardless of whether Oracle Clusterware is running or fully functional.
Note: The OLR is backed up at the end of an installation or an upgrade. After that time, you can only manually back up the OLR. Automatic backups are not supported for the OLR. OCR contains all the information about the cluster resources and permissions that the GI needs to operate. OCR contains information about the resources controlled by Oracle Clusterware, including the following: — ASM disk groups, volumes, file systems and instances.
GI backsup OCR, once every 4 hours. However, if you upgrade from a previous Oracle Clusterware release, then you can continue to use raw or block devices. Mirroring can help prevent OCR from becoming a single point of failure. You should configure OCR in two independent disk groups. Typically this is the work area and the recovery area. It is possible to backup the contents of the OCR online, but restoring this data will almost always result in a service interruption.
Logically OCR represents a tree structure but physically each element of data is stored in a separate 4K physical block. Oracle does not support storing OCR on different storage types simultaneously, such as storing OCR on both oracle asm and a shared file system, except during a migration. You can have up to five OCR locations. However, each successive run must point to a different disk group.
If the original OCR location does not exist, then you must create an empty 0 byte OCR location with appropriate permissions before you run the ocrconfig -add or ocrconfig -replace commands. If asm is used, the asm disk group that you specify should be existing and mounted.
If OCR resides on a cluster file system file or a network file system, create an empty 0 byte OCR location file before performing the procedures in this section. It may be necessary to repair OCR if your cluster configuration changes while that node is stopped and this node is the only member in the cluster. For example, if any node that is part of your current Oracle RAC cluster is shut down, then you must update the OCR configuration on the stopped node to let that node rejoin the cluster after the node is restarted.
This operation only changes OCR on the node on which you run this command. When you repair OCR on a stopped node using ocrconfig -repair, you must provide the same OCR file name which should be case-sensitive as the OCR file names on other nodes. If you run the ocrconfig -add -repair -replace command, then the device, file, or Oracle ASM disk group that you are adding must be accessible. This means that a device must exist. The ocrconfig -replace command requires that at least two OCR locations are configured.
If an OCR location suddenly becomes unavailable, then it might take a short period for Oracle Clusterware to show the change in status. Note: The OCR location that you are replacing can be either online or offline. The preceding command fails if you have less than two configured OCR locations that are online. Note: If your cluster configuration changes while the node on which OCR resides is stopped, and the Oracle Clusterware stack is running on the other nodes, then OCR detects configuration changes and self-corrects the configuration by changing the contents of the ocr.
At any one time, Oracle always retains the last three backup copies of OCR. You cannot customize the backup frequencies or the number of files that Oracle Database retains. The above command displays the backup location, timestamp, and the originating node name of the backup files that Oracle Clusterware creates. By default, the -showbackup option displays information for both automatic and manual backups but you can include the auto or manual flag to display only the automatic backup information or only the manual backup information, respectively.
Note: When the clusterware stack is down on all nodes in the cluster, the backups that are listed by the ocrconfig -showbackup command may differ from node to node. You cannot restore your configuration from an OCR backup file using the -import option, you must instead use the -restore option.
If the original OCR location does not exist, then you must create an empty 0 byte OCR location with the same name as the original OCR location before you run the ocrconfig -restore command. To restore OCR: 1. Ignore any errors that display. Check whether crsd is running.
Run the ocrconfig -repair -replace command as root on all the nodes in the cluster where you did not the ocrconfig -restore command. For example, if you ran the ocrconfig -restore command on node 1 of a four-node cluster, then you must run the ocrconfig -repair -replace command on nodes 2, 3, and 4.
Internet host table DB VIPs Cluster Interconnect 1 Cluster Interconnect 2 ETL Private Network DataDomain My pictorial version. It initializes resources that need elevated OS privileges. It is created by the OHAS daemon. It starts all resources that do not require root access. It was started with release EVMD publishes Oracle Clusterware events to all nodes of the cluster like, ability to start and stio nodes, instances and services. This daemon is intended for backward compatibility.
It is an alternative to NTP server. CSSD has 3 separate processes: 1. CSS Daemon ocssd , 2. CSS Monitor cssdmonitor. ASM disk headers have metadata from This does not require the ASM to be up. Once the voting disks are identified, CSS can access and join the cluster. OCR resource configuration includes definitions of dependencies on other cluster resources, timeouts, retries, assignment and failover policies. Below are started. ACFS daemon. GNS, if available. ACFS registry.
Node VIP. The functionality provided by the Oracle Agent Process in 11gR2 was provided by racgmain and racgimon background processes in earlier releases. Case: Unable to start a instance of a 4 node RAC, after a upgrade from Considerations: 1. Check version of the racgimon process of the database, as it is the mediator of the CRS and the database.
To correct it, 1. Start the new process. Check the new process. ASM diskgroups mounted. Step 3: Restore ocr and voting disk from backup. List backup ocrconfig —showbackup. With the introduction of Clusterware This is especially true for crsctl, which is now the tool to access and manipulate low level resources in Clusterware.
This also implies that some of the notes on Metalink are no longer applicable to Clusterware It has been extended though, and quite dramatically so in If you are interested at which log level a specific Clusterware resource operates, you can use the crsctl get log resource resourceName call, as in this example:.
CRS Command Get failed, or completed with errors. Interestingly, most Clusterware daemons start leaving a detailed message in their log file as to which module uses what logging level. Take CSSD for example:. Now how do you check this on the command line? Back to the question as to which modules make up CSSD, I got this answer from Clusterware, matching the log file output:. Note that the name of the module has to be in upper case. The object I would like to research is the voting file discovery.
I know that the ASM disks have an entry in the header indicating whether or not a disk contains a voting file, in the kfdhdb. With the default log information, I can see the voting disk discovery happening as shown in this cut down version of the ocssd. Does it make a difference? Restarting Clusterware should give more clues. Looking at the logfile I could tell there was no difference. I tried using the trace level now for the same module:.
So I decided to leave it at this point as I was running out of time for this test. On the other hand, I wonder if it is ever necessary to do so. We are going to discuss just about cache fusion. You should have the architecture knowledge about RAC. Also you can visit my previous post about Oracle RAC installation to get some basic information and installation details.
Before Oracle 8i the situation was different. If we take a case of multi-instance Oracle Parallel server and if one of the instance ask for a block of data which is currently modified by other instance of same database, then the holding instance needs to write the data to disk so that requesting instance can read the same data.
This has greatly affected the performance of the database. With Oracle 8i, partial cache fusion was implemented. Following table gives the scenario when cache fusion works in Oracle 8i OPS and scenario where cache fusion was not working. Off course these limitations are not present in Oracle 10g RAC. So when requesting instance ask for a block which is present in holding instance in a read or write mode and if the block is dirtied, then cache fusion used to work and block from cache of holding instance used to get copied to requesting instance.
But if block is not dirtied and block is present in holding instance then requesting instance has to read the block from datafile. Also if the block is opened for write in holding instance and other instance wants to update the same block then holding instance have to write the block to disk so that requesting instance can read it.
Concept of cache fusion Cache Fusion basically is about fusing the memory buffer cache of multiple instance into one single cache. For example if we have 3 instance in a RAC which is using the same datafiles and each instance is having its own memory buffer cache in their own SGA, then cache fusion will make the database behave as if it has a single instance and the total buffer cache is the sum of buffer cache of all the 3 instance.
Below figure shows what I mean. This behaviour is possible because of high speed interconnect existing in the cluster between each instance. Each of instance is connected to other instance using a high-speed interconnect. This makes it possible to share the memory between 2 or more servers.
Previously only datafile sharing was possible, now because of interconnect, even the cache memory can be shared. But how this helps? Well, for example if we have a data block in one of the instance and its updating the block and other instance needs the same data block then this data block can be copied from holding instance buffer cache to requesting instance buffer cache using this high-speed interconnect.
This high speed interconnect is a private connection made just for sending data blocks and more by instances. External users cannot use this connection. It is this interconnect which makes multiple server behave like a cluster. These servers are bind together using this interconnect.
Next we will see how cache fusion works. But before that we need to discuss few important headings which are very important to understand. We will discuss following topics before discussing Cache Fusion. For example another user has already read a block of data and changed some rows in buffer cache. If another user want to read the data from same data block then Oracle will make a copy of that data block in buffer cache and apply the undo information present in undo tablespace to get a consistent view of data.
This consistent data is then presented to user who wants to read the data. This is called maintaining consistency of data. Now consider a multi instance system RAC, where a data block might not be present in same instance. A user might be updating data block in some other instance. If data block are already available in local instance then they will be immediately available to the user.
The current block is the one which contains all the changes, committed as well as uncommitted. Example a user fired a DML on a data block which is not present in any of the instance. Then this block will be read from disk into buffer cache where the value gets changed.
After then user commits and fires another DML on same data block. Now that data block is dirty and contains committed as well as uncommitted changes. Thus we have multiple versions of same data blocks, each of them are consistent with respect to the user who requested. During the course of operation there can be many more version of same data block, each of them consistent with respect to some point in time.
Within the shared cache, co-ordination of concurrent task is called synchronization. The synchronization provided by Oracle RAC provides a cluster wide concurrency of resource and in turn ensure integrity of shared data. All though there is synchronization within the cache, there is some cost involved for doing the same. According to Oracle studies, accessing the block in a local cache is much faster than accessing the block from another instance cache with in the cluster.
Because with local cache is the in memory copy and with other instance cache, the data transfer needs to be done over high speed interconnect which is obviously slower than in memory copy. Worst is the copy from disk, which is much slower than above two processes. This is represented by background process LMSn. There can be max 10 LMS process for an instance. The main function of GCS is to track the status and location of data blocks.
Status of data block means the mode and role of data block. GCS is also responsible for block transfer between the instances. Each block is assigned to a specific instance using the same hash algorithm that is used for global resources. The instance managing the block is known as the resource master. When an instance requires access to a specific block, a request is sent to an LMS process on the resource master requesting access to the block. The LMS process can build a read-consistent image of the block and return it to the requesting instance, or it can forward the request to the instance currently holding the block.
The LMS processes coordinate block updates, allowing only one instance at a time to make changes to a block and ensuring that those changes are made to the most recent version of the block. The LMS process on the resource master is responsible for maintaining a record of the current status of the block, including whether it has been updated.
In Oracle 9. The number of required LMSn processes varies depending on the amount of messaging between the nodes in the cluster. This involves all non-cache fusion intra instance operations. GES performs concurrency control on dictionary cache locks, library cache locks and transactions.
It performs this operation for resources that are accessed by more than once instance. Enqueue services are also present in single instance database. These are responsible for locking the rows on a table using different locking modes. LMON: In a single-instance database, access to database resources is controlled using enqueues that ensure that only one session has access to a resource at a time and that other sessions wait on a first in, first out FIFO queue until the resource becomes free.
In a single-instance database, all locks are local to the instance. In a RAC database there are global resources, including locks and enqueues that need to be visible to all instances. For example, the database mount lock that is used to control which instances can concurrently mount the database is a global enqueue, as are library cache locks, which are used to signal changes in object definitions that might invalidate objects currently in the library cache. In Oracle LMDn : The current status of each global enqueue is maintained in a memory structure in the SGA of one of the instances.
For each global resource, three lists of locks are held, indicating which instances are granted, converting, and waiting for the lock. The LMD background process is responsible for managing requests for global enqueues and updating the status of the enqueues as requests are granted. Each global resource is assigned to a specific instance using a hash algorithm.
When an instance requests a lock, the LMD process of the local instance sends a request to the LMD process of the remote instance managing the resource. If the resource is available, then the remote LMD process updates the enqueue status and notifies the local LMD process. If the enqueue is currently in use by another instance, the remote LMD process will queue the request until the resource becomes available.
It will then update the enqueue status and inform the local LMD process that the lock is available. The LMD processes also detect and resolve deadlocks that may occur if two or more instances attempt to access the two or more enqueues concurrently.
It manages requests for resources other than data blocks —for example, library and row cache objects. LCK processes handle all resource transfers not requiring Cache Fusion. It also handles cross-instance call operations. The files generated by this process can be forwarded to Oracle Support for further analysis. There is one DIAG background process per instance.
It should not be disabled or removed. In the event that the DIAG background process itself fails, it can be automatically restarted by other background processes. GRD is like an in-memory database which contains details about all the blocks that are present in cache. GRD know what is the location of latest version of block, what is the mode of block, what is the role of block etc.
Typically GRD contains following and more information. The modes are as follows: a Null N Mode : Null mode is the least restrictive mode. It indicates no access rights. It acts as a place holder. However another session can read the data block. Other resource cannot have write over this data block. However it can have consistent read on this datablock.
GCS resources also have roles. Following are the different roles present: a Local : When a data block is first read into the instance from the disk it has a local role. Meaning that only 1 copy of data block exists in the cache. No other instance cache has a copy of this block. For example a user connected to one of the instance request for a data block. This data block is read from disk into an instance. The role granted is local. If another instance request for same block this block will get copied to the requesting instance and the role becomes global.
In an Oracle database, a typical block is not written to disk immediately after it is dirtied. This is to reduce excessive IO. When the same dirty block is requested by some other instance for write of read purpose, an image of the block is created in owning instance and then the block is shifted to requesting instance.
This image copy of the block is called Past Image PI. In the event of failure Oracle can reconstruct the block by reading PIs. It is also possible to have more than 1 PI of the block, depending on how many times the block was requested in dirty stage. A past image of the block is different to CR Consistent read image. Past image is required to create CR by applying undo data. The buffer state of a block relates to the access mode of the block.
For example, if a buffer state is exclusive current XCUR , an instance owns the resource in exclusive mode. That is, if the instance holds an older version of the data. There can be only one copy of any one block buffered in the XCUR state in the cluster database at any time.
To perform modifications on a block, a process must assign an XCUR buffer state to the buffer containing the data block. For example, if another instance requests read access to the most current version of the same block, then Oracle changes the access mode from exclusive to shared, sends a current read version of the block to the requesting instance, and keeps a PI buffer if the buffer contained a dirty block.
At this point, the first instance has the current block and the requesting instance also has the current block in shared mode. Therefore, the role of the resource becomes global. There can be multiple shared current SCUR versions of this block cached throughout the cluster database at any time. For explaining this example, I am assuming a 3 node RAC system and also another assumption is that any DML statement is followed by a commit.
But there is no checkpoint until the end. Stage 1 In stage 1 datablock is requested by a user C who is connected to instance 3. So a datablock is read into the buffer cache of instance 3. Assume this gives a value of Stage 2 In stage 2 user B issues the same select statement against the salesman table. Instance 2 will need the same block; therefore, the block is shipped from instance 3 to instance 2 via cache fusion interconnect.
There is no disk read at this time. Here if you see carefully that even though the block is present in more than once instance, still we say that role is local because the block is not yet dirtied. Had the block been dirty and then requested by other instance, then in that case the role will change to global. Stage 3 In stage 3 user B decides to update the row and commit at instance 2.
The new sales rank is Also the role of the block is still LOCAL because the block is dirtied for the first time only on instance 2 and no other instance is having any dirty copy of that. If another instance now tries to update same block the role will change to global.
Stage 4 In stage 4 user A decides to update in instance 1 the same row and hence the same block with salesrank of It finds that block is dirtied in instance 2. Stage 5 User C executes a select statement from instance 3 on same row. The datablock from instance 1 being the most recent copy GRD Global Resource Directory knows this information about which instance is having the latest copy of datablock , it is shipped to instance 3. If the instance 3 asked for exclusive lock then the instance 1 would have had NULL lock.
Stage 6 User B issues the same select statement against the salesman table on instance 2. Instance 2 will request for a consistent copy of buffer from another instance, which happens to be the current master. So instance 2 mode becomes SG1. Stage 7 User C on instance C updates the same row. Further if any instance wants to read or write on the same block then a copy will be again shifted from instance 3.
Diagwait attribute was introduced in It has also been subsequently back ported to This means it is possible to set diagwait on If that is the case one is advised to apply the latest patchset available before attempting to set diagwait. Starting with Diagwait can be set on windows but it does not change the behavior as it does on Unix-Linux platforms.
In Most cases when the node is evicted, there is information written to the logs to analyze the cause of the node eviction. However in certain cases this may be missing, the steps documented in this note are to be used for those cases where there is not enough information or no information to diagnose the cause of the eviction.
It may be useful to set diagwait attribute to delay the node reboot to give additional time to the OS to write the traces. This setting will provide more time for diagnostic data to be collected by safely and will NOT increase probability of corruption. After setting diagwait, the Clusterware will wait an additional 10 seconds Diagwait — reboottime. Customers can unset diagwait after fixing their OS scheduling issues.
Setting diagwait: It is important that the clusterware stack must be down on all the nodes when changing diagwait. If there are clusterware processes running and you proceed to the next step, you will corrupt your OCR. Do not continue until the clusterware processes are down on all the nodes of the cluster.
Check if diagwait is set successfully by executing the following command. The command should return Restart the Oracle Clusterware on all the nodes by executing: crsctl start crs. Validate that the node is running by executing: crsctl check crs.
Diagwait delays the node eviction and reconfiguration by diagwait 13 seconds and as such setting diagwait does not affect most customers. In case there is a need to remove diagwait, the above mentioned steps need to be followed except step 3 needs to be replaced by the following command.
Perform the following configuration procedures on all nodes in the cluster! Oracle9 i Release 1 9. Starting with Oracle9 i Release 2 9. The hang-check timer is loaded into the Linux kernel and checks if the system hangs. It will set a timer and check the timer after a certain amount of time. There is a configurable threshold to hang-check that, if exceeded will reboot the machine.
Although the hangcheck-timer module is not required for Oracle Clusterware Cluster Manager operation, it is highly recommended by Oracle. Hangcheck-timer will provide message logging to the system messages log when a failure is detected, and a node restart is initiated by the module:. The hangcheck-timer. If the system hangs or pauses, the timer resets the node.
TSC is incremented at each clock signal. The TCS offers much more accurate time measurements because this register is updated by the hardware automatically. Hangcheck-timer will not cause reboots to occur due to CPU starvation. Installing the hangcheck-timer. Configuring and Loading the hangcheck-timer Module There are three key parameters to the hangcheck-timer module:. Also note that for the system to be rebooted, it must hang and then recover to a sufficient operational state for the hangcheck-timer module to run.
A permanent hang will not be detected, and the system must be rebooted manually. All hangcheck-timer default values should be explicitly overridden when loading the kernel module, based on the Oracle release as follows:. This configurable value is set when the root. By default, the value is set to one minute. Configuring Hangcheck Kernel Module Parameters Each time the hangcheck-timer kernel module is loaded manually or by Oracle , it needs to know what value to use for each of the two parameters we just discussed: hangcheck-tick and hangcheck-margin.
These values need to be available after each reboot of the Linux server. It is commonly used indirectly: udev relies upon modprobe to load drivers for automatically detected hardware. Jon Masters and others maintain module-init-tools. Manually Loading the Hangcheck Kernel Module for Testing Oracle is responsible for loading the hangcheck-timer kernel module when required. For that reason, it is not required to perform a modprobe or insmod of the hangcheck-timer kernel module in any of the startup files i.
In earlier releases, hangcheck-timer was loaded using insmod in place of modprobe. Consult your release specific documentation to determine which initialization method is required. The hangcheck-timer module will be loaded by Oracle automatically when needed. Although you could load the hangcheck- timer kernel module by passing it the appropriate parameters e. Cluster Interconnect in Oracle 10g and 11g [ID Physical interface including NIC, cables and switches 2.
Private Interconnect used by Oracle Clusterware 3. Physical Layout of the Private Interconnect The basic requirements are described in the Installation Guide for each platform. Additional information about certification can be found on Metalink Certify. The interconnect, as identified by both subnet number and interface name must be configured on all clustered nodes. A switch between the clustered nodes is an absolute requirement. Clusterware uses the interconnect for cluster synchronization network heartbeat and daemon communication between the clustered nodes.
This communication is based on the TCP protocol. Cache Fusion is the remote memory mapping of Oracle buffers, shared between the caches of participating nodes in the cluster. The volume and traffic patterns of this type of data, shared between nodes can vary greatly depending on the applications. There are some vendor specific protocol exceptions in Oracle 10g.
Interconnect Failure The private interconnect is the critical communication link between nodes and instances. Network errors will negatively impact Oracle Clusterware communication as well as RAC communication and performance. A subdivided cluster can result in data corruption; consequently immediate action is taken to resolve this condition.
Interconnect failures; therefore, result in a node or subset of nodes in the cluster shutting down. At this time the Oracle Clusterware happens to use the node numbers to resolve this, but this could change in the future. RAC instances will wait for the end of cluster reconfiguration to start their own reconfiguration. For that purpose the interconnect should be configured across two NICs as well as two switches for complete redundancy so that the cluster can survive a single point of failure.
This means that an underlying failure is handled by the OS or networking drivers that manage the interfaces. Oracle software does not recognize the change underneath, because the failover is handled by the operating system transparently to Oracle. Private Interconnect for Oracle Clusterware The private node name determines the interface being used for Oracle Clusterware and is defined during installation of the Clusterware.
With 3rd party Vendor Clusterware in place Oracle Clusterware should be configured to use the same interconnect often referred to as the heartbeat as the underlying vendor cluster software. This enables you to change the IP addresses when you need to move the server to a different IP range. Private Interconnect for RAC The database uses the private interconnect for communication and cache fusion.
Oracle highly recommends that the database and Oracle Clusterware share the same interconnect. A value in the spfile or init. Identification of the Private Interconnect for RAC The value of the private interconnect for an instance can be identified using 1. This view aims to answer the question on where Oracle found the information about a specific interconnect.
These views are not available in Oracle 10g The alert. Different Interconnects for Clusterware and RAC Oracle Clusterware does not monitor interfaces other than the one specified via the private host name during installation. If RAC instances run with an interconnect that is different from the Oracle Clusterware interconnect any failure will remain undetected by Oracle Clusterware.
The instances will detect this failure themselves and clear the situation by a mechanism called Instance Membership Recovery IMR. Instance evictions due to IMR happen after 10 minutes. This configuration should be avoided. Keep in mind that a failure of one of those interfaces will cause the instance to fail. The failed instance can only be restarted when all the interfaces are fixed or you start all the instances without the faulty interface.
Misconfigurations 1. Physical Interface Sometimes misconfigurations on the physical layer are very hard to identify. You need to make sure that your network components work together and are certified with each other. Private Interconnect for Oracle Clusterware Incorrect private host name used for the cluster interconnect: you need to reinstall Clusterware to get the desired name. References NOTE Having a single name to access the cluster allows clients to use the EZConnect client and the simple JDBC thin URL to access any database running in the cluster, independently of which server s in the cluster the database is active.
SCAN provides load balancing and failover for client connections to the database. The SCAN works as a cluster alias for databases in the cluster. There are 2 options for defining the SCAN:. Three IP addresses are recommended considering load balancing and high availability requirements regardless of the number of servers in the cluster. The IP addresses must be on the same subnet as your public network in the cluster.
The IPs must not be assigned to a network interface on the cluster , since Oracle Clusterware will take care of it. Note: If your DNS server does not return a set of 3 IPs as shown above or does not round-robin, ask your network administrator to enable such a setup.
DNS using a round-robin algorithm on its own does not ensure failover of connections. However, the Oracle Client typically handles this. It is therefore recommended that the minimum version of the client used is the Oracle Database 11g Release 2 client. All Oracle Database 11g Release 2 tools used to create a database e.
However, in order to overcome the installation requirement without setting up a DNS-based SCAN resolution, you can use a hosts-file based workaround. It is not possible to simulate the round-robin resolution that the DNS server does using a local host file. The host file look-up the OS performs will only return the first IP address that matches the name.
Neither will you be able to do so in one entry one line in the hosts-file. Thus, you will create only 1 SCAN for the cluster. Note that you will have to change the hosts-file on all nodes in the cluster for this purpose. This workaround might also be used when performing an upgrade from former pre-Oracle Database 11g Release 2 releases.
The remaining reconfiguration is then performed automatically. During cluster configuration, several resources are created in the cluster for SCAN. In case, a 2-node-cluster is used for which 3 IPs are still recommended for simplification reasons , one server in the cluster will host two sets of SCAN resources under normal operations.
If by means of such a failure the number of available servers in the cluster becomes less than three, one server would again host two sets of SCAN resources. If a node becomes available in the cluster again, the formerly mentioned dispersion will take effect and relocate one set accordingly. This allows the instances to register with the SCAN Listeners as remote listeners to provide information on what services are being provided by the instance, the current load, and a recommendation on how many incoming connections should be directed to the instance.
By default, a node listener is created on each node in the cluster during cluster configuration. The client will then go through the list it receives from the DNS and try connecting through one of the IPs received. If the client receives an error, it will try the other addresses before returning an error to the user or application. This is similar to how client connection failover works in previousreleases when an address list is provided in the client connection string.
It will then re-direct the connection request to the local listener on the node where the least loaded instance is running. Subsequently, the client will be given the address of the local listener. The local listener will finally create the connection to the database instance.
If the version of the Oracle Client connecting to the database as well as the Oracle Database version used are both Oracle Database 11g Release 2 and the default configuration is used as described in this paper, no changes to the system are typically required. The same holds true, if the Oracle Client version and the version of the Oracle Database that this client is connecting.
The disadvantage of this configuration is that SCAN would not be used and hence the clients are still exposed to changes every time the cluster changes in the backend. Similarly, if an Oracle Database 11g Release 2 is used, but the clients remain on a former version. The following cases need to be considered:. Hence, it will try to connect to only the first address returned in the list and will more or less ignore the others.
If you have implemented a Maximum Availability Architecture MAA environment, in which you use Oracle RAC for both your primary and standby database in both, your primary and standby site , which are synchronized using Oracle Data Guard, using SCAN provides a simplified TNSNAMES configuration that a client can use to connect to the database independently of whether the primary or standby database is the currently active primary database.
It specifies the timeout duration in seconds for a client to establish an Oracle Net connection to an Oracle database. Using these two parameters, both, the SCAN on the primary site and the standby site, can be used in the client connection strings. Even, if the randomly selected address points to the site that is not currently active, the timeout will allow the connection request to failover before the client has waited unreasonably long the default timeout depending on the operating system can be as long as 10 minutes.
However, unlike a virtual IP, the SCAN is associated with the entire cluster, rather than an individual node, and associated with multiple IP addresses, not just one address. Because all services on the cluster are registered with the SCAN listener, the SCAN listener replies with the address of the local listener on the least-loaded node where the service is currently being offered. Finally, the client establishes connection to the service through the listener on the node where service is offered.
All of these actions take place transparently to the client without any explicit configuration required in the client. During installation listeners are created. Oracle Net Services routes application requests to the least loaded instance providing the service. Because the SCAN addresses resolve to the cluster, rather than to a node address in the cluster, nodes can be added to or removed from the cluster without affecting the SCAN address configuration.
At a minimum, the SCAN must resolve to at least one address. Otherwise, it defaults to clustername-scan. For example, if you start Oracle grid infrastructure installation from the server node1, the cluster name is mycluster, and the GNS domain is grid. To enable GNS, you must have your network administrator provide a set of IP addresses for a subdomain assigned to the cluster for example, grid. The set of IP addresses is provided to the cluster through DHCP, which must be available on the public network for the cluster.
Prior to installing Oracle CRS, the raw devices that will house the OCR and voting disk files, must be defined and initialized using the following procedure. Additionally, this section does not describe how to setup Linux multipathing, if multipathing is employed, then use the appropriate the multipath device name in place of the devices listed below.
Run fdisk command to create and label MB partition for each the OCR and voting disk you plan on deploying. Initialize the devices using dd. For each raw device that you specified in the rawdevices file, enter commands similar to the following to set the owner, group, and permissions on the device file.
The root. List the appropriate disk names as requested. Note this procedure uses raw partitions for these files and not shared-concurrent logical volumes. Determine which disk device will be used store the OCR file and voting disk. Make sure that the devices chosen have PVIDs on them. Note, the disk and its PVID must be visible and consistent between nodes.
If the device name associated with the PVID for a disk that you want to use is different on any node, you must create a new device file for the disk on each of the nodes using a common unused name. In this case, create a dummy disk; e. This will ensure the disk name is the same across all the nodes. To allow concurrent IO access to this disk device and prevent the device driver from locking the hdisks with a reservation on open, a no reservation flag must be set. Use the following chdev command to disable this reservation.
Change the permissions and ownership on the two devices. List the appropriate disk device paths as listed in Step 1. Note, the following procedures are only necessary if the devices in question appear as different device names SCSI paths. If the device names are same, then skip to step 5. Execute ioscan -fnC disk command on all nodes of cluster to obtain the hardware path and Instance number for the selected devices. On node2: ioscan -fnC disk This will generate output similar to the following note the differing devices name and Instance numbers between the two nodes.
Prior to installing Oracle CRS, raw devices that will house the OCR and voting disk files, must be defined and initialized using the following procedure:. Determine which disk devices will be used store the OCR and voting disk file. Use the format command to list the devices, and select the appropriate device. Make sure the other nodes can see those devices. If the other nodes see those devices at a different path, then common symbolic links or mknod command must be used to point to the same location.
CFS provides single-image view of devices; therefore, the CRS files can be located on the cluster filesystem. Determine the path and name of the OCR and voting disk. Proceses connected with this service are evmd runs as root while evmd. This process also starts the racgevt process to manage FAN server callouts.
Cluster Synchronization Services CSS Manages the cluster configuration by controlling which nodes are members of the cluster and by notifying members when a node joins or leaves the cluster. If you are using third-party clusterware, then the css process interfaces with your clusterware to manage node membership information. Processes connected to this service are init.
Failure of this process results in cluster restart. Anything that the crs process manages is known as a cluster resource which could be a database, an instance, a service, a Listener, a virtual IP VIP address, an application process, and so on. This includes start, stop, monitor and failover operations. The crs process generates events when a resource status changes. When you have installed RAC, crs monitors the Oracle instance, Listener, and so on, and automatically restarts these components when a failure occurs.
By default, the crs process makes five attempts to restart a resource and then does not make further restart attempts if the resource does not restart. Process connected with this service is crsd. This process restarts automatically upon failure.
If no additional trace file was generated, this field is blank. This metric contains the fully specified name of the current XML alert log file including directory path. This information is useful for diagnostic purposes because it can be used to correlate related problems encountered by a single user attempting to accomplish a single task.
Diagnostic incidents usually indicate software errors and should be reported to Oracle through the Enterprise Manager Support Workbench. This metric provides an optional field that reports the impact of the problem that occurred.
It may be empty. This field is informational. Diagnostic incidents usually indicate software errors and should be reported to Oracle using the Enterprise Manager Support Workbench. This metric reports the incident ID, a number that uniquely identifies a diagnostic incident a single occurrence of a problem.
A problem is one or more occurrences of the same incident. If you use Support Workbench, the incident ID can be used to select the correct problem to package and send to Oracle. This metric reports the number of Generic Incident type incidents observed the last time that Oracle Enterprise Manager scanned the alert log.
This metric reflects the number of Generic Internal Error incidents observed the last time Enterprise Manager scanned the alert log. This metric category contains metrics representing errors that might affect the operation of the database as recorded in the database alert log file. The alert log file has a chronological log of messages and errors. This metric reports the number of generic operation errors observed the last time Enterprise Manager scanned the alert log file.
This metric reports the number of user-defined errors observed the last time Enterprise Manager scanned the alert log file. This metric reflects the number of user-defined warnings witnessed the last time Enterprise Manager scanned the alert log file. The metrics in this category provide information about the Oracle high availability service alert log. This metric tracks and raises an alert when a resource changes to a state defined in the thresholds.
CRS Event Manager is online. In this case, start cluster as follows. But Oracle High Availability Service should be online for the following command to succeed. Unknown August 3, at PM. Newer Post Older Post Home. Subscribe to: Post Comments Atom. This error means that you are trying to perform some operation in the database which requires encryption wallet to be open, but wallet is Tablespace Growth History and Forecast for 10g and 11g. Finding space usage of tablespaces and database is what many DBAs want to find.
In this article I will explain how to find out space usage Manual upgrading Oracle database 11g to 12c. For upgrading
But Oracle High Availability Service should be online for the. I followed my dreams and Sign up using Google. Sign up or log in got demoted to software developer. Opt-in alpha test for a. Finding space usage of tablespaces badges 19 19 silver badges 25 25 bronze badges. PARAGRAPHIn this case, start cluster. In this article I will explain how to find out space usage Manual upgrading Oracle. Unknown August 3, at PM. Sign up using Email and. Visual design changes to the.start mount 2nd standby RAC instance got controlfile header error. Posts: CRS Oracle High Availability Services is online. CRS Cluster I bet one of those steps I did wrong, which caused issue. You have to put in. CRS Oracle High Availability Services is online. CRS Cannot communicate with Cluster Ready Services. CRS Cluster. CRS Shutdown of Oracle High Availability Services-managed resources on CRS Oracle High Availability Services is online.