How can you tell if LV is being (local disk) Mirrored or Not? 1

[KOG]

Hi folks,

$ lspv hdisk1
PHYSICAL VOLUME: hdisk1 VOLUME GROUP: oracledata
PV IDENTIFIER: 00016001e0441e02 VG IDENTIFIER 000160016f674e9d
PV STATE: active
STALE PARTITIONS: 0 ALLOCATABLE: yes
PP SIZE: 64 megabyte(s) LOGICAL VOLUMES: 2
TOTAL PPs: 135 (8640 megabytes) VG DESCRIPTORS: 2
FREE PPs: 116 (7424 megabytes)
USED PPs: 19 (1216 megabytes)
FREE DISTRIBUTION: 27..08..27..27..27
USED DISTRIBUTION: 00..19..00..00..00

If you look at this there are 166 PP's free @ 64Mb each = 7.5Mb (Approx) still free on this harddisk that hasn't been allocated as yet. My df shows

/dev/lv00 20054016 363768 99% 19148 1% /opt

So opt appears to still have about 9.5Gb space allocated though, which correctly means that I am not using the full amount of both of the hard disks here (just over 1 is allocated). I could increase the number of PP's to add extra space. You've got 116 free at 64mb each = 7.25 mb.

$ lspv hdisk2
PHYSICAL VOLUME: hdisk2 VOLUME GROUP: oracledata
PV IDENTIFIER: 000160016f32d004 VG IDENTIFIER 000160016f674e9d
PV STATE: active
STALE PARTITIONS: 0 ALLOCATABLE: yes
PP SIZE: 64 megabyte(s) LOGICAL VOLUMES: 1
TOTAL PPs: 135 (8640 megabytes) VG DESCRIPTORS: 1
FREE PPs: 0 (0 megabytes)
USED PPs: 135 (8640 megabytes)
FREE DISTRIBUTION: 00..00..00..00..00
USED DISTRIBUTION: 27..27..27..27..27

This disk appears to be used looking at the all free PP being used.

$ lspv -l hdisk1
hdisk1:
LV NAME LPs PPs DISTRIBUTION MOUNT POINT
loglv00 1 1 00..01..00..00..00 N/A
lv00 18 18 00..18..00..00..00 /opt

$ lspv -l hdisk2
hdisk2:
LV NAME LPs PPs DISTRIBUTION MOUNT POINT
lv00 135 135 27..27..27..27..27 /opt

$ lslv lv00
LOGICAL VOLUME: lv00 VOLUME GROUP: oracledata
LV IDENTIFIER: 000160016f674e9d.2 PERMISSION: read/write
VG STATE: active/complete LV STATE: opened/syncd
TYPE: jfs WRITE VERIFY: off
MAX LPs: 512 PP SIZE: 64 megabyte(s)
COPIES: 1 SCHED POLICY: parallel
LPs: 153 PPs: 153
STALE PPs: 0 BB POLICY: relocatable
INTER-POLICY: minimum RELOCATABLE: yes
INTRA-POLICY: middle UPPER BOUND: 32
MOUNT POINT: /opt LABEL: /opt
MIRROR WRITE CONSISTENCY: on
EACH LP COPY ON A SEPARATE PV ?: yes

Or was the fact that the /opt filesystem was only copied across two disks?

I am a bit confused here, should I run the following command to mirror the LV00 onto hdisk1?

i.e.

mklvcopy LV00 2 hdisk1 (is that correct?)

Thanking you all in advance

Regards

Katherine

 

[PSD]

Katherine,

Basically you cannot mirror the lv00 logical volume as you do not have the room, lv00 spans accross hdisk1 and hdisk2. hdisk2 is full and hdisk1 has the jfslog 1 PP and 18 PP's of lv00 as they are both the same size of disk the math does not add up.

If you lose any disk hdisk1 or hdisk2 you will lose your filesystem data on that disk and be forced to replace the disk and restore from backup. The best way forward is to use hdisk1 to expand the filesystem and put in two more disks so you can set up 1-1 mirroring meaning a disk failure will not take down /opt.

I hope this is not a HA system!!!

Best of luck

PSD
IBM Certified Specialist - AIX V4.3 Systems Support
IBM Certified Specialist - AIX V4 HACMP

 

[KOG]

Hi

I had the consultant here to help me with the installation of oracle and I think he made a mistake in 'mirroring' the filesystem /opt rather than LV00. The main /opt is on hdisk2 not hdisk1 but hdisk1 has a 'copy' of /opt which is rather confusing to me.

I thought I have enough space to mirror LV00?

I have three PVs on this VG and I plan to create two more LVs and spread the datafiles that are on /opt across three PVs and mirror them.

 

[PSD]

Katherine,

OK just a few fundamentals to start with, if hdisk1 and hdisk2 are both in oracledata vg then the filesystem is not mirrored. You mirror the LV not a filesystem, /opt is just a mount point addressing all of the data contained with lv00 so you mirror at the LV level not the filesystem level. If it were mirrored your lsvg and lslv outputs would be significantly different i.e.:

lslv lv00 :

copies = 1 - this would read copies = 2.

lsvg -l oracledatavg

lv00 if the LP's and PP's columns are the same then the data is not mirrored, i.e. 1 LP = 1 PP, mirroring would mean that for every 1 LP there would be 2 PP, so it would be double the number of LP's in a two way LV copy scenario.

I think you need to write down exactly what you plan to do, but here is why you cannot mirror over three disk disks currently.

hdisk2 and hdisk1 - lv00 has a total of 153 LP's.

Each disk only has 135 LP's so /opt is split over two disks, leaving only one to mirror to you are 18 LP's short (this assumes that the 3rd disk is the same size.)

I hope that clears things up.

PSD
IBM Certified Specialist - AIX V4.3 Systems Support
IBM Certified Specialist - AIX V4 HACMP

 

[KOG]

Hi PSD,

Is it possible to make TWO copies of /opt filesystem? As looking at this output

$ lsvg -l oracledata
oracledata:
LV NAME TYPE LPs PPs PVs LV STATE MOUNT POINT
loglv00 jfslog 1 1 1 open/syncd N/A
lv00 jfs 153 153 2 open/syncd /opt

And df shows that /opt is

$ df
Filesystem 512-blocks Free %Used Iused %Iused Mounted on
/dev/hd4 65536 55320 16% 955 6% /
/dev/hd2 786432 30032 97% 13156 14% /usr
/dev/hd9var 229376 212440 8% 146 1% /var
/dev/hd3 983040 947744 4% 72 1% /tmp
/dev/hd1 32768 31488 4% 53 2% /home
/dev/lv00 20054016 6822368 66% 19139 1% /opt
/dev/lv01 16384000 15869592 4% 25 1% /backup

The questions is HOW did /opt filesystem spread across TWO disks? That is what I am so confused about? Did the consultant when creating /opt filesystem set it to two copies? Or set it to spread across TWO disks?

The following is documentation of my plan and I hope I am following in the right direction?

We had an oracle consultant on site few weeks ago to assist me with the installation of oracle on new server F50 on AIX 4.3.3. All was seems ok as he was only here for short time and I was expected to carry on with this work.

I have created four VGs – rootvg, oracledata and backup.

At first I created the oracledata vg using PP = 64mb which is rather big as the disk is 9gb in size as I had problems trying to create it using smitty (later found out there is a fast command for creating vg using large disk).

There are three PVs within oracledata vg (hdisk1, hdisk2 & hdisk4).

Now the outputs of all disks within oracledata vg

$ lsvg oracledata
VOLUME GROUP: oracledata VG IDENTIFIER: 000160016f674e9d
VG STATE: active PP SIZE: 64 megabyte(s)
VG PERMISSION: read/write TOTAL PPs: 405 (25920 megabytes)
MAX LVs: 256 FREE PPs: 251 (16064 megabytes)
LVs: 2 USED PPs: 154 (9856 megabytes)
OPEN LVs: 2 QUORUM: 2
TOTAL PVs: 3 VG DESCRIPTORS: 3
STALE PVs: 0 STALE PPs: 0
ACTIVE PVs: 3 AUTO ON: yes
MAX PPs per PV: 2032 MAX PVs: 16

Hard Disk 1

$ lspv hdisk1
PHYSICAL VOLUME: hdisk1 VOLUME GROUP: oracledata
PV IDENTIFIER: 00016001e0441e02 VG IDENTIFIER 000160016f674e9d
PV STATE: active
STALE PARTITIONS: 0 ALLOCATABLE: yes
PP SIZE: 64 megabyte(s) LOGICAL VOLUMES: 2
TOTAL PPs: 135 (8640 megabytes) VG DESCRIPTORS: 1
FREE PPs: 116 (7424 megabytes)
USED PPs: 19 (1216 megabytes)
FREE DISTRIBUTION: 27..08..27..27..27
USED DISTRIBUTION: 00..19..00..00..00

hdisk1:
PP RANGE STATE REGION LV ID TYPE MOUNT POINT
1-27 free outer edge
28-28 used outer middle loglv00 jfslog N/A
29-36 free outer middle
37-54 used outer middle lv00 jfs /opt
55-81 free center
82-108 free inner middle
109-135 free inner edge

$ lspv -l hdisk1
hdisk1:
LV NAME LPs PPs DISTRIBUTION MOUNT POINT
loglv00 1 1 00..01..00..00..00 N/A
lv00 18 18 00..18..00..00..00 /opt

Hard Disk 2

$ lspv hdisk2
PHYSICAL VOLUME: hdisk2 VOLUME GROUP: oracledata
PV IDENTIFIER: 000160016f32d004 VG IDENTIFIER 000160016f674e9d
PV STATE: active
STALE PARTITIONS: 0 ALLOCATABLE: yes
PP SIZE: 64 megabyte(s) LOGICAL VOLUMES: 1
TOTAL PPs: 135 (8640 megabytes) VG DESCRIPTORS: 1
FREE PPs: 0 (0 megabytes)
USED PPs: 135 (8640 megabytes)
FREE DISTRIBUTION: 00..00..00..00..00
USED DISTRIBUTION: 27..27..27..27..27

$ lspv -p hdisk2
hdisk2:
PP RANGE STATE REGION LV NAME TYPE MOUNT POINT
1-27 used outer edge lv00 jfs /opt
28-54 used outer middle lv00 jfs /opt
55-81 used center lv00 jfs /opt
82-108 used inner middle lv00 jfs /opt
109-135 used inner edge lv00 jfs /opt

$ lspv -l hdisk2
hdisk2:
LV NAME LPs PPs DISTRIBUTION MOUNT POINT
lv00 135 135 27..27..27..27..27 /opt


Hard Disk 4

$ lspv hdisk4
PHYSICAL VOLUME: hdisk4 VOLUME GROUP: oracledata
PV IDENTIFIER: 00016001145eb54e VG IDENTIFIER 000160016f674e9d
PV STATE: active
STALE PARTITIONS: 0 ALLOCATABLE: yes
PP SIZE: 64 megabyte(s) LOGICAL VOLUMES: 0
TOTAL PPs: 135 (8640 megabytes) VG DESCRIPTORS: 1
FREE PPs: 135 (8640 megabytes)
USED PPs: 0 (0 megabytes)
FREE DISTRIBUTION: 27..27..27..27..27
USED DISTRIBUTION: 00..00..00..00..00

$ lspv -p hdisk4
hdisk4:
PP RANGE STATE REGION LV ID TYPE MOUNT POINT
1-27 free outer edge
28-54 free outer middle
55-81 free center
82-108 free inner middle
109-135 free inner edge

$ lspv -l hdisk4
$

What concerns me is the way the consultant set up the filesystem and I think he made a copy of it on hdisk1, if you look at the outputs

$ lspv -l hdisk2
hdisk2:
LV NAME LPs PPs DISTRIBUTION MOUNT POINT
lv00 135 135 27..27..27..27..27 /opt

$ lspv -l hdisk1
hdisk1:
LV NAME LPs PPs DISTRIBUTION MOUNT POINT
loglv00 1 1 00..01..00..00..00 N/A
lv00 18 18 00..18..00..00..00 /opt

The first one is full and the second one is only about 20% full, does this mean it does not hold the full copy of /opt filesystem or was it spread across two disks? I thought you can only do local mirror of LV not Fs?

QUESTION: How can I tell if there is TWO copies of /opt filesystem? Is there a command? Is there anyway I can break the copy of filesystem /opt? Or should I leave it at this state? Is it a good idea to make copy of fs or perform local mirroring of LV?

The plan is to do the following tasks as I am not happy with the way the consultant configured the disk, he put ALL the oracle datafiles within /opt filesystem on ONE disk (hdisk2) !!

Plan:

1. To create LV called LV01 on hdisk1 using the following statistics (as would like to ensure enough space for mirroring other LVs on the same disk).

Free space on hdisk1 = 7424 PP size is 64 mb

For LV01, the LV Partitions = (4000/64)= 62 (I can always increase it later on).

I will use this LV01 to create filesystem /oracledata1 and move some datafiles from /opt on hdisk2 onto this filesystem. The files I will move are

Control02.ctl
RBS01
Redo1b
Redo2b
Redo3b
Users01

This ensures I have enough space for mirroring another LV from other disk i.e. mirroring LV00 from hdisk2 to hdisk1 using the following command

#mklvcopy lv00 2 hdisk1 (is that correct)?

What do you think?

2. Create another LV called LV02 on hdisk4 using the following statistics.

Free space on hdisk1 = 8640 PP size is 64 mb

For LV02, the LV Partitions = (4000/64)= 62 (I can always increase it later on).

I will use this LV02 to create filesystem /oracledata2 and move some datafiles from /opt on hdisk2 onto this filesystem. The files I will move are

Control03.ctl
Redo1a
Redo2a
Redo3a
Temp01
Tools01

This ensures I have enough space for mirroring another LV from other disk i.e. mirroring LV01 from hdisk1 to hdisk4 using the following command

#mklvcopy lv01 2 hdisk4 (is that correct)?

The only problem I have no is not having enough space to mirror LV02, but I do have another disk and I plan to use it for backup. Until a new slot cage is installed on the F50 to install another 9gb disk.

 

[KOG]

Hi again,

$ lslv lv00
LOGICAL VOLUME: lv00 VOLUME GROUP: oracledata
LV IDENTIFIER: 000160016f674e9d.2 PERMISSION: read/write
VG STATE: active/complete LV STATE: opened/syncd
TYPE: jfs WRITE VERIFY: off
MAX LPs: 512 PP SIZE: 64 megabyte(s)
COPIES: 1 SCHED POLICY: parallel
LPs: 153 PPs: 153
STALE PPs: 0 BB POLICY: relocatable
INTER-POLICY: minimum RELOCATABLE: yes
INTRA-POLICY: middle UPPER BOUND: 32
MOUNT POINT: /opt LABEL: /opt
MIRROR WRITE CONSISTENCY: on
EACH LP COPY ON A SEPARATE PV ?: yes

From the output above for LV00 what does

1) MIRROR WRITE CONSISTENCY: ON mean?

2) EACH LP COPY ON A SEPARATE PV?: YES

I assume from the outputs above it states mirroring is on and that a copy of LV is made on another disk ??? If I am wrong then this information seems misleading?

 

[PSD]

Katherine,

Firstly the lv is not mirrored it just spans two disk as the first one filled up and someone increased the size of the filesystem and it allocated free LP's from hdisk2. The lslv output you posted say COPIES = 1, this means that it not mirrored, as for MWCC that is an option for all LV's, by default it is on and only comes into play when the LV is mirrored.

In terms of you plan you cannot mirror the /opt filesystem in a three disk scenarion as you just do not have the space, as lv00 is not mirrored you need four disks in the volume group to mirror it properly, or clean down /opt and remove - recreate it in a smaller size. If you think about it if the data spans two disks and there is not enough room to put it just on one then you will need two disks to mirror to. I sugest you get another disk ordered!!



PSD
IBM Certified Specialist - AIX V4.3 Systems Support
IBM Certified Specialist - AIX V4 HACMP

 

[Yegolev]

If I may (good job PSD!)...

The EACH LP COPY ON A SEPARATE PV option is useful if you are planning on putting your PPs on separate disks, which is a really good idea. If that option is not on, the LVM will indeed create 2 PP for each LP however it will just stick them wherever it can, sometimes on the same physical disk. Bothersome if you want to change it later, so leave that option on.

PSD pointed out that your disks are 135PP capacity and lv00 is 153PP, so lv00 was distributed to a second pv. This sort of thing is why volume groups are useful to keep data contained.

This next part is optional! =)

What you might want to do now, for performance, is reorg lv00 so that it resides on the edge of the disks. Later, when you have another pair of disks for the mirror, you will be able to mirror with the same layout (if I remember right). From `smit chlv` choose Change Logical Volume, pick your lv, then:

* Logical volume NAME emcd11
Logical volume TYPE [jfs]
POSITION on physical volume middle
RANGE of physical volumes minimum
MAXIMUM NUMBER of PHYSICAL VOLUMES [32]
to use for allocation
Allocate each logical partition copy yes
on a SEPARATE physical volume?
RELOCATE the logical volume during yes
reorganization?
Logical volume LABEL [/opt]
MAXIMUM NUMBER of LOGICAL PARTITIONS [512]
SCHEDULING POLICY for reading/writing parallel
logical partition copies
PERMISSIONS read/write
Enable BAD BLOCK relocation? yes
Enable WRITE VERIFY? no
Mirror Write Consistency? yes

The option "POSITION on physical volume" is set to middle right now, the other options are center, inner-middle, inner-edge, edge. Edge is the outer part of the platter which spins fastest so your data should be out there.

The option "RANGE of physical volumes" should be set to maximum before you add any additional drives because that will spread your data across all the disks evenly. Combined with the above option will put as much data on the edge as possible. You would want to have just two disks in your datavg and set this option, then reorg the vg (using reorgvg command). Afterwards you can add two more disks and mirror everything. IBM Certified -- AIX 4.3 Obfuscation