Tek-Tips is the largest IT community on the Internet today!
Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!
-
Congratulations Mike Lewis on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!
MSF900 Archiving
- Thread starter rc068
- Start date
We have tested MSB907 in 1997 as at MIMS3.10 and the results were not encouraging, in particular we found that roll-back of archived transactions did not appear to work correctly. You can expect that by now the problems would have been fixed, still it is worthwhile testing.
This site is now at MIMS 4.1 and has been reluctant to undertake any type of archiving due to the lack of confidence in the robustness of the MIMS archiving solutions. It still has all transactions on MSF900. Note that as process date in Julian format is stored on 4 digits, I think we shall see a Y2k style issue in 2007-2008 when the values in the field will reach 9999 and Mincom will need to redesign the data model and ammend all software involved, inc MSSDAT and MSB907(just an added issue).
Glad you didn't mention archiving to the final step of MSF078; that would be a definite NO, you may as well use sql and delete the transactions, that useful MSF078 is. Note however that MSB907 may archive to MSF078 depending on the values you specify on request screen. The major issue with MSF078 is that it requires un-archiving in case you do not accept to lose transactions when you upgrade MIMS.
Some of the potential problems I could see at the stage when I tested MSB907 were:
- MSB907 inefficient with long execution time impacting on the overnight and possibly requiring restarting; restart procedure not explored and risk of leaving database in inconsistent state. It took 11 hours in order to redistribute two periods containing approx. 70,000 transactions between MSF900-MSF901-MSF902.
The MSB907 parameter "Maximum duration of run in minutes" may be used to limit the execution time. This may suggest that the program may take an excessively long time to execute. The documentation indicates that if a time limit is specified and the program suspends its execution without completing the redistribution process, the request for MSB907 "will remain active and it will continue when next allowed to run". Clarifications need to be sought in regards to:
* which is the current status of the files after MSB907 has suspended execution
* can MIMS be made available to the users(new transactions being added) and after a number of days, can execution of MSB907 be resumed?
* how does the user initiate the program to resume execution
- MSB907 had failed to transfer all transactions for a financial period, ending up with a period's transactions split among files
- all your custom reports and data extracts in RDL or other tools that read MSF900, must be changed to also read MSF901 and MSF902, if allowed to report on historical records; they will become less efficient
- The MSB907 request parameter "Do Consistency Check", quoting from the MIMS manual, "will only typically be required when a previous run of this process has not produced the expected redistribution results". This may be interpreted as an acknoledgement that MSB907 may fail, leaving the database in an inconsistent state
It is not clear if re-running MSB907 with "do consistency check"="Y" will provide diagnosis only, or will also fix any problems; and how much longer would it take to execute with this option selected?
- depending on your platform, database sizing issues need to be addressed before running MSB907, ie removing 100,000 records from msf900 and moving them to MSF901, the tables must be properly sized; test what happens with the database if the process fails due to various causes, such as time-out or table space allocation - is the database ending up inconsistent?
Note that Keyword Serach "archive" will take you to other related threads in this forum.
This site is now at MIMS 4.1 and has been reluctant to undertake any type of archiving due to the lack of confidence in the robustness of the MIMS archiving solutions. It still has all transactions on MSF900. Note that as process date in Julian format is stored on 4 digits, I think we shall see a Y2k style issue in 2007-2008 when the values in the field will reach 9999 and Mincom will need to redesign the data model and ammend all software involved, inc MSSDAT and MSB907(just an added issue).
Glad you didn't mention archiving to the final step of MSF078; that would be a definite NO, you may as well use sql and delete the transactions, that useful MSF078 is. Note however that MSB907 may archive to MSF078 depending on the values you specify on request screen. The major issue with MSF078 is that it requires un-archiving in case you do not accept to lose transactions when you upgrade MIMS.
Some of the potential problems I could see at the stage when I tested MSB907 were:
- MSB907 inefficient with long execution time impacting on the overnight and possibly requiring restarting; restart procedure not explored and risk of leaving database in inconsistent state. It took 11 hours in order to redistribute two periods containing approx. 70,000 transactions between MSF900-MSF901-MSF902.
The MSB907 parameter "Maximum duration of run in minutes" may be used to limit the execution time. This may suggest that the program may take an excessively long time to execute. The documentation indicates that if a time limit is specified and the program suspends its execution without completing the redistribution process, the request for MSB907 "will remain active and it will continue when next allowed to run". Clarifications need to be sought in regards to:
* which is the current status of the files after MSB907 has suspended execution
* can MIMS be made available to the users(new transactions being added) and after a number of days, can execution of MSB907 be resumed?
* how does the user initiate the program to resume execution
- MSB907 had failed to transfer all transactions for a financial period, ending up with a period's transactions split among files
- all your custom reports and data extracts in RDL or other tools that read MSF900, must be changed to also read MSF901 and MSF902, if allowed to report on historical records; they will become less efficient
- The MSB907 request parameter "Do Consistency Check", quoting from the MIMS manual, "will only typically be required when a previous run of this process has not produced the expected redistribution results". This may be interpreted as an acknoledgement that MSB907 may fail, leaving the database in an inconsistent state
It is not clear if re-running MSB907 with "do consistency check"="Y" will provide diagnosis only, or will also fix any problems; and how much longer would it take to execute with this option selected?
- depending on your platform, database sizing issues need to be addressed before running MSB907, ie removing 100,000 records from msf900 and moving them to MSF901, the tables must be properly sized; test what happens with the database if the process fails due to various causes, such as time-out or table space allocation - is the database ending up inconsistent?
Note that Keyword Serach "archive" will take you to other related threads in this forum.
I would advise that you determine why you are going to archive the MSF900 before going forward. We assumed that archiving the MSF900 would bring about performance improvements in MIMS and were disappointed with the results.
At the time, we were running 3.013 and had about 20,000,000 records in the MSF900. After spending nearly a month archiving the transactions in small chunks, this was reduced to less than 6,000,000 in the 900 with the others spread between the 901 and the 902. The performance impact was unmeasureable.
I believe that there may be some justification in 'archiving' the 900 in terms of database recoverability, but don't assume (as I did) that you will improve performance.
If you think that the size of the MSF900 may be impacting your performance, make sure that you don't have any 'old unposted' transactions that are forcing un-needed processing in the MSB900 runs.
Glen Colbert
gcolbert@mslden.com
At the time, we were running 3.013 and had about 20,000,000 records in the MSF900. After spending nearly a month archiving the transactions in small chunks, this was reduced to less than 6,000,000 in the 900 with the others spread between the 901 and the 902. The performance impact was unmeasureable.
I believe that there may be some justification in 'archiving' the 900 in terms of database recoverability, but don't assume (as I did) that you will improve performance.
If you think that the size of the MSF900 may be impacting your performance, make sure that you don't have any 'old unposted' transactions that are forcing un-needed processing in the MSB900 runs.
Glen Colbert
gcolbert@mslden.com
- Status
