Hi folks, I am a bit concerned a 1

[KOG]

Hi folks,

I am a bit concerned about the oracledata VG I have created and I think it is down to the way it has been configured at the start using smitty to create VG. Now after reading some intensive notes on VG I believe there is a command mkvg -t to adjust the size of VG and the max allocation of PPs within the VG.

What I am a bit concerned is I have already installed oracle and datafiles on the first disk and there are now THREE disks within oracledata VG. According to the VG limitation chart, it stated

No of Disks Max no of PPs/Disk
1 13048
2 65024
4 32512

Now I have a problem it only states those divisble by twos, what if I have three disks within VG (I have a spare one and plan to use it for backups only). Should I allocate the new disk to Oracledata VG and change the PP settings using mkvg -t command?

It is all news to me and I am not 100% certain what tasks I should follow but at the moment the first disks is already full and it is 9gb disk (the total size of our db is only 3gb !!!). So I need to be able to spread the datafiles across several disks but want to be able to maximise the use of all disks (as cost is the key issue in this company !)

Hope you can help me please.

Thanking you all in advance.

Regards

Katherine


VOLUME GROUP: oracledata VG IDENTIFIER: 000160016f674e9d
VG STATE: active PP SIZE: 64 megabyte(s)
VG PERMISSION: read/write TOTAL PPs: 405 (25920 megabytes)
MAX LVs: 256 FREE PPs: 251 (16064 megabytes)
LVs: 2 USED PPs: 154 (9856 megabytes)
OPEN LVs: 2 QUORUM: 2
TOTAL PVs: 3 VG DESCRIPTORS: 3
STALE PVs: 0 STALE PPs: 0
ACTIVE PVs: 3 AUTO ON: yes
MAX PPs per PV: 2032 MAX PVs: 16

oracledata:
LV NAME TYPE LPs PPs PVs LV STATE MOUNT POINT
loglv00 jfslog 1 1 1 open/syncd N/A
lv00 jfs 153 153 2 open/syncd /opt

oracledata:
PV_NAME PV STATE TOTAL PPs FREE PPs FREE DISTRIBUTION
hdisk1 active 135 116 27..08..27..27..27
hdisk2 active 135 0 00..00..00..00..00
hdisk4 active 135 135 27..27..27..27..27

 

[Chapter11]

What you'll want to do is this:

[tt]chlv -e x lv00[/tt]

That command will alter the allocation policy for that logical volume (lv00) to be distributed across the maximum range of physical volumes (default is 32, but since you have 3, it'll distributed the LV across 3). After that:

[tt]reorgvg oracledata lv00[/tt]

That command will reorganize the logical volume "lv00" using its new allocation policy. You can do this online with the system running without shutting down Oracle. Disk I/O performance will go to hell for a while, but Oracle can stay up.

Your oracledata volume group was already created with some non-default settings. Normally, you cannot have more than 1016 partitions on a physical volume (a hard disk), but it is possible to increase that limit, of course at the cost of something else.

You can double the allowable number of physical partitions on a disk by halving the number of physical disks allowed in a volume group. The normal values are 1016 PPs-per-PV and 32 PVs-per-VG. In your case, this balance has been adjusted to 2032 PPs-per-PV at a cost of lowering your max PVs for that VG to 16. There are some good reasons for doing so, but given the info on that particular VG, I see no reason to rebuild it right now with different specs, unless you have planned growth matters that will change your requirements over time.

 

[KOG]

Hi Chapter11

Pardon me if I am slow but this concept is rather confusing. Now I have three 9gb disks within oracledata VG. I have already set the PP settings for each PV, PP is 64 mb. I do not know if this is the best PP size but it is as far as I could go esp with the size of the disk.

So if I created large PP for each PV, is that economical or should I try and create smaller PP as possible i.e. 32 mb instead of 64? What difference will it make. Oracledata vg is important and I know I will add one or more disks in the near future so that I can spread the datafiles across the disks within vg.

So the size of LP is dependable on the size of PP already created?

Does having large PP cause disk to fill up quickly? I have deleted all trace, dump, logs etc files that I can think of but for some reason the disk seems to fill up quickly and I can't think of any other reason?

Thanking you all in advance.

Cheers

Katherine

 

[Chapter11]

AIX's LVM takes a little getting used to if you come from another background in terms of disk management, or no background at all.

I'll try to explain it.

A Volume Group (VG) contains Physical Volumes (PV). A Physical Volume is usually the same as a hard disk (a PV can be many hard disks if you're using a RAID controller, but we'll leave that alone for now).

When you add a hard disk / PV to a VG, AIX divides the space on the hard disk into many Physical Partitions (PP). Under normal conditions, the maximum number of PPs that can exist on a PV is 1016.

If your PP size is set to 1 meg, then a PV would be limited in useful size to about 1 gig. Larger PP sizes allow for larger useful PV's, always multiplying by that factor of 1016. If you have a PP size of 64 megs, then the largest disk you can use usefully is roughly 64 gigs.

This also means that if you have a PP size of 1 meg, and you add a 2 gig hard drive to the VG, you will only be able to use 1 gig of that drive, wasting half of it. As I recall, AIX will either warn you, or simply disallow this.

The PP size for a VG is set when the VG is created and cannot be changed dynamically later.

When a new hard disk is added to a VG, the PPs that are created are put in a pool of PPs. You have 3 PVs in your oracledata VG, so the PPs that were created on all of those drives are put in one large pool (the VG).

Now we cross a line from physical to logical.

A Logical Partition (LP) is mostly identical to a Physical Partition (PP). For your basic system, 1 LP is going to map directly to 1 PP. When you get into mirroring, that becomes untrue, but the analogy will suffice for the moment.

A Logical Volume (LV) is a collection of Logical Partitions. Each LP in the LV maps directly to a PP which is a physical location on a hard drive.

Once you have your LV created, you can put a filesystem on top of it, and store files in your filesystem.

Going back down the layers: you access a file in a filesystem. The filesystem is stored in a Logical Volume. The LV is made up of 1 or more Logical Partitions. Each LP maps directly to a Physical Partition, and each PP is actual space on the hard drive.

It's a lot of layers, yes, and it'll take you some getting used to (I recommend taking IBM's AIX V4 Sys Admin class, at least the section on the LVM). There's some good reasons for having so many layers between a filesystem and physical disk space, as compared to most other operating systems: once you get used to it all, you can do some really slick things.

It's just a matter of time and exposure. I don't know about the other people on this board, but the majority of the time I spend doing AIX stuff is disk space management.

Addressing some of your questions regarding PP size:

A smaller PP size will usually give slightly more efficient use of the physical volumes than a larger PP size. If you use a PP size of 4 meg, then a new disk added to the VG can have up to just-under 4 megs of wasted space. If you were to use a 1 meg PP size, then you'd have less than a meg of wasted space, utilizing 3 megs that you weren't using before. Since this scales up in powers of two, there are cases where you can waste quite a bit of space.

One LP/PP is the minimum allocatable unit. If you need to create a filesystem to hold 32 megs worth of data, and your PP/LP size is 256 megs, you have to create a 256 meg filesystem, you can't create one smaller than that. In such a case, yes you do waste some disk space.

At this point I feel like I'm rambling, so I think I should stop until you can review this and tell me if it makes sense.

As for the last issue, your filesystems filling up quickly: that's going to be an Oracle issue - AIX and the LVM don't cause filesystems to fill up on their own. I'm not a DBA, but you might check to see if your database is in archive log mode. Archive logs can eat disk space quickly.

 

[KOG]

Morning Chapter11,

Many thanks for all the wonderful info, it has cleared up alot of issues regarding to LV and PP (took a while to work it out as I think it is rather complicate storage issue of AIX).

So I will test using the commands you have given

chlv -1 x lv00 (will it automatically read the number of PVs within oracledata VG?)


on the 43P (already installed AIX 4.3.3 and Oracle and it has the same configuration and structures as F50).

Then run

reorgvg oracledata lv00

and see what happens. I personally would like to manually move the datafiles across three disks.

As you are aware the size of PP in oracledata is 64mg, do you think this is one of the main reason for fast filesystem or logical volume growth? As I do constantly clear up all the dump files, trace files, audit files, logs and archived files. Also I run the find command to find any recently used files and check them etc.

So if the oracledata PP is 64mb, have I wasted a lot of space in the PVs?

And again many thanks.

Regards

Katherine

 

[Chapter11]

You got a typo.

[tt]chlv -e x lv00[/tt]

This will change the allocation policy for logical volume "lv00" to use the maximum number of disks. The number of PVs to spread the logical volume across isn't quite important yet - you could set that here, but it defaults to 32 which is just fine for your situation.

I personally would like to manually move the datafiles across three disks.

That's exactly what the [tt]reorgvg oracledata lv00[/tt] command does. It doesn't give you any progress reports though, so all you can do is wait. If you have topas installed, you can watch the disk levels as the data is moved around.

With your PP size set to 64M, you are wasting at most 63 megs of space per disk (probably less), which is pretty insignificant. I wouldn't worry about that too much. You have to completely rebuild your VG if you want to change the PP size, so that little disk space is rarely worth the effort it takes to rebuild a VG.

The PP size doesn't have anything to do with how fast a filesystem fills up, that's going to be an application issue. My best suggestion there is to acquire the services of an Oracle DBA and have them take a look at your database to see if it's doing anything it doesn't need to be doing.

And finally, yes, AIX's LVM is not the simplest way of managing disk space, but once you get used to it, it's absolutely beautiful. I don't even like thinking about working on a system without a decent volume manager!

 

[KOG]

HI Chapter11

I have never heard of topas?

I tried the commands you suggested on 43P and it seems to run fine. When querying to see if /opt filesystem and lv00 logical volume has spread across three PVs on Oracledata VG. But I would like to know exactly which disks the oracle datafiles are located, is there any commands to assist me?

Also I did some experiment I created two LVs, one on each new disks within Oracledata VG (now have LV00, LV01 and LV02) and two new filesystems one each for LV01 and LV02. So there is one LV and filesystem on each disk. I am aware that /opt and lv00 has been migrated across three disks. Moved some datafiles from /opt filesystem to the new filesystems.

This is just a test to see how performance works and how the space is used. Is it a good idea to create more LVs within the same volume group so that I can move the datafiles around rather than using them all within one filesystem i.e. /opt?

Will this degrade the o/s and oracle performance?

Regards

Katherine

 

[Chapter11]

You can get topas by installing the perfagent.tools filesets from your installation media. It's similar to the more generic "monitor", and gives a host of real-time performance information constantly updated.

The command: "lslv -l lv00" will give you a list of physical volumes that the given logical volume is located on.

It is completely normal to have many logical volumes in a volume group - having a volume group that contains only one logical volume (not counting the jfslog) and thus one filesystem is very unusual in my (limited) experience.

Tuning a system to run an Oracle database is a pretty deep subject, and depends on knowing how the database is accessed as well as the underlying technology. Since there are multiple components to an Oracle database that have different access characteristics (actual data, indexes, redo logs, archive logs, and so on), it takes some time, research, and empirical collection of data to get the "best" tuning you can get.

The simplest way to start that I can think of is to distribute all Oracle data across all possible disks. There are of course negatives to doing this, but doing so at least spreads all of the positives and negatives across everything. Maybe in the future you'll find that having a particular piece of Oracle data is better served by sticking it on a disk by itself. That's easy enough to change in the future, and shouldn't be difficult to do with the system online and hot. Reorganizing logical volumes across physical volumes with the system online and running is one of those "slick" things I mentioned earlier =)

Once you get more and more familiar with AIX's LVM and disk management in general, you'll become more effective at tuning logical volumes/filesystems for performance.

 

[KOG]

Hi

MAny thanks for all your help and the info you have given me is absolutely brillaint, it has cleared up a lot of LVM concepts.

There is one more questions I would like to ask you and I promise this is the final ques.

Is it worth creating paging space for oracledata VG i.e.

paging01 on the third disk of oracledata VG, size of paging01 will be 2048mb. Will this improve the performance if it is located within VG?

Thanking u in advance.

Regards

Katherine

 

[Chapter11]

My personal recommendation is to keep all paging spaces in rootvg.

If for some reason, another volume group that contains a paging space goes offline, you'll crash.

Also, if you have multiple paging spaces, try to keep them equal in size.

 

[KOG]

Many thanks again.

Cheers

Katherine

 

[Yegolev]

Note that adding more paging space will slow down your machine a bit if all other factors are OK. For better performance you should keep as much data in RAM as possible, then there is the overhead associated with maintaining a larger paging space. Adding paging is really just a way to get around a memory constraint.

Somewhat related, you want to keep your Oracle SGA smaller than your RAM, otherwise you are defeating the purpose of having a SGA. IBM Certified -- AIX 4.3 Obfuscation