Job Prioritization acting odd 1

[GMACTom]

Does this Job Cost Priority system work as advertised in ver 7.2.1?

I select project, then menu option Administration, select Prioritization. In wizard, check prioritization by cost and enter range values 0-400, 401-800, 801-999 for Light, Med Heavy assigned as Light=High Priority, Med=Medium Priority, Heavy=Low Priority.

Then create 6 test reports, right click select priority tab and enter a cost: 2 @ priority cost 200, 2 @ 600, and 2 @ 900. Select all, send to history, switch to job monitor, and keep hitting refresh. Low priority jobs execute, then medium, then high priority jobs last.

Is this right? Have I set up something backward, or do I need to update schema (no), or what? Am I just stupid to think a low priority job should wait for a high priority job? In dev, governing setting is down to 1 executing job per user in hope this will make testing easier. Is this jamming things up in the MicroStrategy scheduler?

This was going to be a one hour exercise to set up, now I don't know what to believe.

As our management has chosen to drop all support, this forum is my only resource. Many thanks to those who can help.

 

[nlim]

no support....bad luck. You might need to try 2 things.

1) delete the cached database connections.
2) restart the server.

 

[GMACTom]

This of course had no effect.

I have my six test jobs on dev server so I switched priority of HIGH and LOW. Selected all, sent to history, them displayed active jobs. MED priority jobs were running, others had finished. (I had deleted history and cache both before this test.)

Anyone have suggestions about my methodology? Something here is behaving improperly.

 

[entaroadun]

The functionality is working as advertised. Once a job starts, it will execute until finished; it doesn't stop in order for a higher priority job to execute.

When you send all six to history, they all pile up on the I-Server in order received. The first job received is low, and there are low slots available, so it executes. The other 5 have to wait. The I-Server doesn't reorder the 5 to let the high jobs thru first. When the first is done, the second, which is low, will execute.

This is actually a good thing. Let's say the high jobs always go thru first. What would happen if high jobs constantly came in? Then low and medium jobs would never get executed because there would always be a high job ahead of it in line. You don't want low jobs to wait forever, you just want to make sure that in general, high jobs aren't tied up waiting for low jobs. You can afford to let some of the low jobs thru while focusing the lion's share of the resources on the high jobs.

There's a CS term for this; I think it's thread starvation, I'm not sure... In any case, you don't want low priority threads to starve, you just want to make sure that high priority threads get most of the food.

Prioritization isn't meant to make sure that high jobs are executed first. It is meant to make sure that in general, high jobs don't wait as long as low and medium jobs to get run.

 

[GMACTom]

Your description leads me to think that if I dropped in 20 or 30 jobs in a high/med/lo priority mix that they would execute in whatever order they were presented to the queue. I am thus somewhat at a loss as to what the "priority" of any report affects.

The situation I am trying to address is as follows. A large DataMart is loaded monthly from Oracle. Aggregate tables are built. Various test reports and stand alone SQL are executed to verify that load is correct and complete. (This has saved us more than once!) Mart is now officially "loaded and open." I fire an event to start a schedule and about 60 user reports line up and begin to run. Good. We want people to use our mart and the more scheuled reports they run the better. But some reports take 10 minutes to run, and some take 6 hours to run. I can serve more people better if I run the short quick reports first and make the long running reports wait until later. (We have several users who insist on pulling line item detail data.) This is why I tried to work with the priority feature, as a way of ordering job execution sequence in the "Mart Monthly" schedule. Does not seem to work this way.

Can you suggest any other way to accomplish this result? I thought about have 3 scheds, but no user will want to subscribe to the "long running, low priority" schedule.

 

[entaroadun]

Priority is for maintaining open slots for concurrent jobs. By default, every project is configured to allow 2 low jobs, 2 medium jobs, and 2 high jobs, for a total of six concurrent jobs.

If you launch two low jobs, they take up the low slots. If you try to launch another low job, it has to wait since there are no low slots. If you launch a medium or high job, it takes up a medium slot.

If instead, you launch two high jobs, they take up the low slots. Jobs take the lowest slot available. If you try to launch a low job, it has to wait, since there are no low slots available. If you launch a medium or high job, it takes up a medium slot.

In this scenario, you could potentially have 6 high jobs running at a time. You could only have 4 medium jobs running, since the two high slots are reserved for high jobs. You could have 2 low jobs, since low jobs can't use the medium or high slots.

Priority only affects concurrency management. In your situation, you could have 2 of the 6-hour jobs running at the same time, but no more. The 2 high slots would always be reserved for the 10-minute jobs.

Don't try to make reports run faster by single-threading them. With Oracle, you get much better performance by running queries concurrently. That's what makes Oracle scalable; it holds back resources IN CASE another request comes in. If you run one report at a time, there will always be CPUs and memory segments unused by Oracle because the system is reserving them just in case another query comes in. Oracle's superior scalability is a game.

Ask your DBA how many concurrent jobs to allow, and set MSI to use them. The priority scheme you've set will work, just not in the way you imagined. On average, high priority reports will end up getting executed first.

 

[GMACTom]

Right, I see how this works. I set up a situation in Dev environment and observed the process. Will review with staff & DBA the most effective way to accomplish user needs.
Thank you for you sticking with me until I got it.