Teradata I/O

[craig322]

I am trying to improve the i/o of my teradata database. The controller activity appears to be very skewed even though my data is evenly distributed across all amps. The following is a snapshot of the activity on my controllers.

Node CtlId Reads Writes
1-04 00200000 12,951,253 5,747,904
00100000 65,691 303,684
00110000 12,957,010 5,772,862
0020100 0 0
00111000 6 0
1-05 00100000 1,242 150,216
00111000 13,121,519 5,871,400
00201000 13,037,500 5,749,715
00110000 0 0
00200000 0 0

Any suggestions on what might cause this would be greatly appreciated.

 

[tdatgod]

Hi,
How does this correspond to your disk arary controllers and /dev/pdisk assignments.

looking at the numbers for 00100000 I would think this corresponds to your system disk which doesn't enter into the query time.

Everybody else is doing 13M/6M and that looks very balanced. 13M/6M over a typical 10 minute logging period is 20K/10K Io's per second.

If the IO size on average is 4 K this would be 80M/20M per second across each controller. Have you run SAR? What does your WAIT I/O look like?

Maybe if you think this is too much I/O for your query maybe you can add indexes and/or collect statistics on the tables so the optimizer can have a few more choices in how to satisfy the query.

but from what I can see in the report this is a very well balanced system.



If your system if cabled in the normal manor the same controller on each node is assigned to a single array.
therefore c110 on 1-04 and c110 on 1-05 map to the same disk array and therefore the same set of vprocs.


--------------------------------------------------------
NOTE:

Everything from here down is speculation based upon on what I think a typical 2 node configuration looks like

( 8 amps per node )
0 - 7 on node 1-04 and
8 - 15 on nodes 1-05

If your configuration isn't like this then change the numbers accordingly.

----------------------------------------------------------

All the AMPS for that array are again typically assigned to one node.

This is because even though we support shared SCSI we only use that for fail over mode ( unless you have the latest SYMBIOS arrays which are cabled a little differntly. ) WE typically don't want IO's from both nodes in a DAISY CHAIN configuration going down the same SCSI BUS. We had some problems with node starvation in the earlier days of Teradata, so we settled on that configuration.

therefore
1-04 110 is the Primary controller for AMPS 0 - 3
1-04 200 is the Promary Controller for AMPS 4 - 7
1-05 110 is the BUDDY NODE controller for AMPS 0 - 3
1-05 200 is the BUDDY NODE controller for AMPS 4 - 7

1-04 111 is the BUDDY node controller for AMP 8 - 11
1-04 201 is the BUDDY node controller for AMP 12 - 15
1-05 111 is the Primary controller for AMP 8 - 11
1-05 201 is the Primary controller for AMP 12 - 15

therefore you will see all the IO's on 1-4 110 for amps 0 - 3 and 1-05 110 for amps 0 - 3 should be 0 unless we fail over ( and maybe during resets ). And conversely all the IO's for 8-11 on 1-05 111 and 1-04 111 for amp 8-11 should be 0. ( or close to 0. )


Now with the Latest SYMBIOS arrays they actually have 4 Independent ports in the back of them, so rather than daisy chain the node together we go straight in from each node.

In this configuration you want half the AMPs assigned to this array assigned to each node ( in this case 0 / 2 to one node and 1 / 3 to the other ).




Hope this helps.