Tek-Tips is the largest IT community on the Internet today!

Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!

  • Congratulations gkittelson on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!

How to prevent overflow (and precision loss) in big factorials

Status
Not open for further replies.

GerritGroot

Technical User
Nov 3, 2006
291
ES
Hi,

I need to compute a series of factorials for high numbers, however, for very high numbers, normal factorials easily break down.

Let me firts post my code

Code:
PROGRAM FactorialTest
IMPLICIT NONE
REAL :: Func1,Func2
WRITE(*,*)Func1(10,5),Func2(10,5)
WRITE(*,*)Func1(20,10),Func2(20,10)
END PROGRAM FactorialTest


REAL FUNCTION Func1(n,alfa)
IMPLICIT NONE
INTEGER, INTENT(IN) :: n,alfa
INTEGER :: Factorial
Func1=REAL(Factorial(n))/(Factorial(alfa)*Factorial(n-alfa))
END FUNCTION Func1


REAL FUNCTION Func2(n,alfa)
IMPLICIT NONE
INTEGER, INTENT(IN) :: n,alfa
INTEGER :: i
REAL :: sum
sum=0.
DO i=0,alfa-1,1
   sum=sum+LOG((n-i)/(i+1.))
END DO
Func2=EXP(sum)
END FUNCTION Func2


RECURSIVE FUNCTION Factorial(n)  RESULT(Fact)
IMPLICIT NONE
INTEGER :: Fact
INTEGER, INTENT(IN) :: n
IF(n<=0)THEN
   Fact=1
ELSE
   Fact=n*Factorial(n-1)
END IF
END FUNCTION Factorial

and the outcome

Code:
     252.000000      252.000000
      11.659760   184755.900000

The original function I got is function 1 (Func1):

Code:
n!/(alfa!*(n-alfa)!)

In order to prevent overflow I wrote that as function 2 (Func2):

Code:
Product(from i=0 upto alfa-1) of { (n-i)/(i+1) }

Which allows me to extend it a bit further for bigger n using the natural LOG, to avoid overflow (see code above).

Code:
n=10 and alfa=5  gives =>    252 both functions work well
n=20 and alfa=10 gives => 184756 only the second function approximates reasonably within +/-0.1 due to the mathematical trick I used.

However, I need to extend it even further, for higher n, upto 100000 for example without losing much precision.

Is there another mathematical trick to prevent overflow without losing much precision then the one with the LOG that I already used?

Thanks,

Gerrit

P.S.
- I don't want to get to double precision and all that, as I don't see that as a real solution
- Approximations for the factorial, like the Stirling one, don't really prevent overflow.
 
Not sure if this helps, but I mean something like this:

gamma_factorial.f95
Code:
[COLOR=#a020f0]program[/color] gamma_factorial
  [COLOR=#2e8b57][b]implicit[/b][/color] [COLOR=#2e8b57][b]none[/b][/color]
[COLOR=#2e8b57][b]  real[/b][/color] :: x
  [COLOR=#2e8b57][b]integer[/b][/color] :: k
  [COLOR=#804040][b]do[/b][/color] k [COLOR=#804040][b]=[/b][/color] [COLOR=#ff00ff]1[/color], [COLOR=#ff00ff]35[/color]
    x [COLOR=#804040][b]=[/b][/color] [COLOR=#008080]real[/color](k)
    [COLOR=#804040][b]write[/b][/color]([COLOR=#804040][b]*[/b][/color],[COLOR=#804040][b]*[/b][/color]) x, [COLOR=#ff00ff]'! = '[/color], factorial(x)
  [COLOR=#804040][b]end do[/b][/color]   
[COLOR=#a020f0]contains[/color]
[COLOR=#2e8b57][b]  real[/b][/color] [COLOR=#a020f0]function[/color] factorial(x)
    [COLOR=#2e8b57][b]implicit[/b][/color] [COLOR=#2e8b57][b]none[/b][/color]
[COLOR=#2e8b57][b]    real[/b][/color], [COLOR=#2e8b57][b]intent[/b][/color]([COLOR=#2e8b57][b]in[/b][/color]) :: x
    factorial [COLOR=#804040][b]=[/b][/color] [COLOR=#008080]gamma[/color](x [COLOR=#804040][b]+[/b][/color] [COLOR=#ff00ff]1.0[/color])
  [COLOR=#a020f0]end function[/color] factorial
[COLOR=#a020f0]end program[/color] gamma_factorial


Output:
Code:
$ g95 gamma_factorial.f95 -o gamma_factorial

$ gamma_factorial
 1. ! =  1.
 2. ! =  2.
 3. ! =  6.
 4. ! =  24.
 5. ! =  120.
 6. ! =  720.
 7. ! =  5040.
 8. ! =  40320.
 9. ! =  362880.
 10. ! =  3628800.
 11. ! =  3.99168E+7
 12. ! =  4.790016E+8
 13. ! =  6.227021E+9
 14. ! =  8.717829E+10
 15. ! =  1.3076744E+12
 16. ! =  2.092279E+13
 17. ! =  3.556874E+14
 18. ! =  6.4023735E+15
 19. ! =  1.21645105E+17
 20. ! =  2.432902E+18
 21. ! =  5.109094E+19
 22. ! =  1.1240007E+21
 23. ! =  2.5852017E+22
 24. ! =  6.204484E+23
 25. ! =  1.551121E+25
 26. ! =  4.0329146E+26
 27. ! =  1.0888869E+28
 28. ! =  3.0488835E+29
 29. ! =  8.841762E+30
 30. ! =  2.6525285E+32
 31. ! =  8.2228384E+33
 32. ! =  2.6313083E+35
 33. ! =  8.683318E+36
 34. ! =  2.952328E+38
 35. ! =  +Inf
 
Yeah I've been told that on a math forum, but what's the advantage of that? Gamma(n+1)=just the same as=n!. Apart from that, if you replace the Factorial by the Gamma function in Func1, it will give a higher precision on lower numbers, but still break down on higher numbers, while func2 still works, with Func2(100,50) for example.
 
What you try to compute seems to be combinatorial number (or binomial coefficient):
C(n, k) = n!/((n-k)! k!)

Instead of using the above formula with factorial, try to use Recursive formula:
C(n, 0) = 1
C(n, k) = C(n-1, k-1) + C(n-1, k)
or Multiplikative formula - see
 
Thanks, your routine doesn't seem to be more precise. It also seems very slow. I also tried:

Code:
RECURSIVE FUNCTION Combinatorial2(n,k) RESULT(C)
IMPLICIT NONE
REAL :: C
INTEGER, INTENT(IN) :: n,k
IF(k==0)THEN
   C=1
ELSE IF(n==0.AND.k>0)THEN
   C=0
ELSE
   C=(REAL(n)/k)*Combinatorial2(n-1,k-1)
END IF
END FUNCTION Combinatorial2

which can be found on the same wiki page, but, though a bit faster, still doesn't beat Func2 from above. (Maybe I should just use a table or so of 10000x10000)
 
Two modifications in your code:

-> in real function func 1 declare factorial to be real
-> in recursive function factorial declare Fact to be real.

then the output of your program is

Code:
252.0000   252.0000
184756.0   184755.9

Norbert


The optimist believes we live in the best of all possible worlds - the pessimist fears this might be true.
 
GerritGroot said:
... still doesn't beat Func2 from above ...

I don't think that func2 is very good. I compared it with function which simply uses the multiplication formula. For example for n = 1000 it delivers +Infinity
Code:
 combi( 10 , 1 ) = 10.
 func2( 10 , 1 ) = 10.
 combi( 10 , 2 ) = 45.
 func2( 10 , 2 ) = 45.000003814697266
 combi( 10 , 3 ) = 120.
 func2( 10 , 3 ) = 120.00000762939453
 combi( 10 , 4 ) = 210.
 func2( 10 , 4 ) = 209.99996948242188
 combi( 10 , 5 ) = 252.
 func2( 10 , 5 ) = 251.99996948242188
 combi( 10 , 6 ) = 210.
 func2( 10 , 6 ) = 209.99996948242188
 combi( 10 , 7 ) = 120.
 func2( 10 , 7 ) = 120.00000762939453
 combi( 10 , 8 ) = 45.
 func2( 10 , 8 ) = 45.000003814697266
 combi( 10 , 9 ) = 10.
 func2( 10 , 9 ) = 10.
 combi( 10 , 10 ) = 1.
 func2( 10 , 10 ) = 1.

 combi( 1000 , 0 ) = 1.
 func2( 1000 , 0 ) = 1.
 combi( 1000 , 10 ) = 2.634095604619702E+23
 func2( 1000 , 10 ) = 2.6340970553978324E+23
 combi( 1000 , 20 ) = 3.394828113024576E+41
 func2( 1000 , 20 ) = +Inf
 combi( 1000 , 30 ) = 2.429608192173745E+57
 func2( 1000 , 30 ) = +Inf
 combi( 1000 , 40 ) = 5.55974423571664E+71
 func2( 1000 , 40 ) = +Inf
 combi( 1000 , 50 ) = 9.460461017585218E+84
 func2( 1000 , 50 ) = +Inf
 combi( 1000 , 60 ) = 1.9742748621859835E+97
 func2( 1000 , 60 ) = +Inf
 combi( 1000 , 70 ) = 7.040360174623101E+108
 func2( 1000 , 70 ) = +Inf
 combi( 1000 , 80 ) = 5.432697287307054E+119
 func2( 1000 , 80 ) = +Inf
 combi( 1000 , 90 ) = 1.0823545595185519E+130
 func2( 1000 , 90 ) = +Inf
 combi( 1000 , 100 ) = 6.385051192630514E+139
 func2( 1000 , 100 ) = +Inf
 combi( 1000 , 200 ) = 6.617155560659308E+215
 func2( 1000 , 200 ) = +Inf
 combi( 1000 , 300 ) = 5.428250046406142E+263
 func2( 1000 , 300 ) = +Inf
 combi( 1000 , 400 ) = 4.9652723862542264E+290
 func2( 1000 , 400 ) = +Inf
 combi( 1000 , 500 ) = 2.702882409454363E+299
 func2( 1000 , 500 ) = +Inf
 combi( 1000 , 600 ) = 4.9652723862542264E+290
 func2( 1000 , 600 ) = +Inf
 combi( 1000 , 700 ) = 5.428250046406142E+263
 func2( 1000 , 700 ) = +Inf
 combi( 1000 , 800 ) = 6.617155560659308E+215
 func2( 1000 , 800 ) = +Inf
 combi( 1000 , 900 ) = 6.385051192630514E+139
 func2( 1000 , 900 ) = +Inf
 combi( 1000 , 1000 ) = 1.
 func2( 1000 , 1000 ) = 1.0000128746032715

Here is the source code I used
Code:
[COLOR=#a020f0]program[/color] combinatorial
  [COLOR=#2e8b57][b]integer[/b][/color] :: n, k

  n [COLOR=#804040][b]=[/b][/color] [COLOR=#ff00ff]10[/color]
  [COLOR=#804040][b]do[/b][/color] k [COLOR=#804040][b]=[/b][/color] [COLOR=#ff00ff]1[/color], n
    [COLOR=#804040][b]write[/b][/color] ([COLOR=#804040][b]*[/b][/color],[COLOR=#804040][b]*[/b][/color]) [COLOR=#ff00ff]'combi('[/color], n, [COLOR=#ff00ff]','[/color], k, [COLOR=#ff00ff]') ='[/color], combi(n,k)
    [COLOR=#804040][b]write[/b][/color] ([COLOR=#804040][b]*[/b][/color],[COLOR=#804040][b]*[/b][/color]) [COLOR=#ff00ff]'func2('[/color], n, [COLOR=#ff00ff]','[/color], k, [COLOR=#ff00ff]') ='[/color], func2(n,k)
  [COLOR=#804040][b]end do[/b][/color]
  [COLOR=#804040][b]write[/b][/color] ([COLOR=#804040][b]*[/b][/color],[COLOR=#804040][b]*[/b][/color]) 

  n [COLOR=#804040][b]=[/b][/color] [COLOR=#ff00ff]1000[/color]
  k[COLOR=#804040][b]=[/b][/color][COLOR=#ff00ff]0[/color]
  [COLOR=#804040][b]do[/b][/color] [COLOR=#804040][b]while[/b][/color] (k [COLOR=#804040][b]<=[/b][/color] n)
    [COLOR=#804040][b]write[/b][/color] ([COLOR=#804040][b]*[/b][/color],[COLOR=#804040][b]*[/b][/color]) [COLOR=#ff00ff]'combi('[/color], n, [COLOR=#ff00ff]','[/color], k, [COLOR=#ff00ff]') ='[/color], combi(n,k)
    [COLOR=#804040][b]write[/b][/color] ([COLOR=#804040][b]*[/b][/color],[COLOR=#804040][b]*[/b][/color]) [COLOR=#ff00ff]'func2('[/color], n, [COLOR=#ff00ff]','[/color], k, [COLOR=#ff00ff]') ='[/color], func2(n,k)
    [COLOR=#804040][b]if[/b][/color] ((k [COLOR=#804040][b]<[/b][/color] [COLOR=#ff00ff]100[/color]) [COLOR=#804040][b].or.[/b][/color] (k [COLOR=#804040][b]>[/b][/color] [COLOR=#ff00ff]900[/color])) [COLOR=#804040][b]then[/b][/color]
      k [COLOR=#804040][b]=[/b][/color] k [COLOR=#804040][b]+[/b][/color] [COLOR=#ff00ff]10[/color]
    [COLOR=#804040][b]else[/b][/color]
      k [COLOR=#804040][b]=[/b][/color] k [COLOR=#804040][b]+[/b][/color] [COLOR=#ff00ff]100[/color]
    [COLOR=#804040][b]end if[/b][/color]
  [COLOR=#804040][b]end do[/b][/color]

[COLOR=#a020f0]contains[/color]
  [COLOR=#2e8b57][b]double precision[/b][/color] [COLOR=#a020f0]function[/color] combi(n, k)
    [COLOR=#0000ff]! multiplication formula[/color]
    [COLOR=#2e8b57][b]implicit[/b][/color] [COLOR=#2e8b57][b]none[/b][/color]
    [COLOR=#2e8b57][b]integer[/b][/color], [COLOR=#2e8b57][b]intent[/b][/color]([COLOR=#2e8b57][b]in[/b][/color]) :: n, k
    [COLOR=#2e8b57][b]integer[/b][/color] :: i, kk

    [COLOR=#804040][b]if[/b][/color] ((k [COLOR=#804040][b]<[/b][/color] [COLOR=#ff00ff]0[/color]) [COLOR=#804040][b].or.[/b][/color] (k [COLOR=#804040][b]>[/b][/color] n) [COLOR=#804040][b].or.[/b][/color] (n [COLOR=#804040][b]==[/b][/color] [COLOR=#ff00ff]0[/color])) [COLOR=#804040][b]then[/b][/color] 
      combi [COLOR=#804040][b]=[/b][/color] [COLOR=#ff00ff]0[/color]
      [COLOR=#804040][b]return[/b][/color]
    [COLOR=#804040][b]end if[/b][/color]

    [COLOR=#804040][b]if[/b][/color] ((k [COLOR=#804040][b]==[/b][/color] [COLOR=#ff00ff]0[/color]) [COLOR=#804040][b].or.[/b][/color] (k[COLOR=#804040][b]==[/b][/color]n)) [COLOR=#804040][b]then[/b][/color]
      combi [COLOR=#804040][b]=[/b][/color] [COLOR=#ff00ff]1[/color]
      [COLOR=#804040][b]return[/b][/color]
    [COLOR=#804040][b]end if[/b][/color]

    [COLOR=#0000ff]! C(n,k) = C(n,n-k)[/color]
    [COLOR=#804040][b]if[/b][/color] (k [COLOR=#804040][b]>[/b][/color] n [COLOR=#804040][b]-[/b][/color] k) [COLOR=#804040][b]then[/b][/color]
      kk [COLOR=#804040][b]=[/b][/color] n [COLOR=#804040][b]-[/b][/color] k
    [COLOR=#804040][b]else[/b][/color]
      kk [COLOR=#804040][b]=[/b][/color] k
    [COLOR=#804040][b]end if[/b][/color]

    combi [COLOR=#804040][b]=[/b][/color] [COLOR=#ff00ff]1[/color]
    [COLOR=#804040][b]do[/b][/color] i [COLOR=#804040][b]=[/b][/color] [COLOR=#ff00ff]1[/color], kk
      [COLOR=#0000ff]!combi = (n - kk + i) * combi / i[/color]
      combi [COLOR=#804040][b]=[/b][/color] (n [COLOR=#804040][b]-[/b][/color] i [COLOR=#804040][b]+[/b][/color] [COLOR=#ff00ff]1[/color]) [COLOR=#804040][b]*[/b][/color] combi [COLOR=#804040][b]/[/b][/color] i
    [COLOR=#804040][b]end do[/b][/color]
  [COLOR=#a020f0]end function[/color] combi

  [COLOR=#2e8b57][b]double precision[/b][/color] [COLOR=#a020f0]FUNCTION[/color] Func2(n,alfa)
    [COLOR=#2e8b57][b]IMPLICIT[/b][/color] [COLOR=#2e8b57][b]NONE[/b][/color]
    [COLOR=#2e8b57][b]INTEGER[/b][/color], [COLOR=#2e8b57][b]INTENT[/b][/color]([COLOR=#2e8b57][b]IN[/b][/color]) :: n,alfa
    [COLOR=#2e8b57][b]INTEGER[/b][/color] :: i
[COLOR=#2e8b57][b]    REAL[/b][/color] :: [COLOR=#008080]sum[/color]
    [COLOR=#008080]sum[/color][COLOR=#804040][b]=[/b][/color][COLOR=#ff00ff]0[/color].
    [COLOR=#804040][b]DO[/b][/color] i[COLOR=#804040][b]=[/b][/color][COLOR=#ff00ff]0[/color],alfa[COLOR=#804040][b]-[/b][/color][COLOR=#ff00ff]1[/color],[COLOR=#ff00ff]1[/color]
       [COLOR=#008080]sum[/color][COLOR=#804040][b]=[/b][/color][COLOR=#008080]sum[/color][COLOR=#804040][b]+[/b][/color][COLOR=#008080]LOG[/color]((n[COLOR=#804040][b]-[/b][/color]i)[COLOR=#804040][b]/[/b][/color](i[COLOR=#804040][b]+[/b][/color][COLOR=#ff00ff]1[/color].))
    [COLOR=#804040][b]END DO[/b][/color]
    Func2[COLOR=#804040][b]=[/b][/color][COLOR=#008080]EXP[/color]([COLOR=#008080]sum[/color])
  [COLOR=#a020f0]END FUNCTION[/color] Func2
[COLOR=#a020f0]end program[/color] combinatorial
 
func2 ... For example for n = 1000 it delivers +Infinity
It was bwcause I forgot to change the declaration from
Code:
REAL :: sum
to
Code:
double precision :: sum
With double precision func2 works.
 
Thank you all for your answers. I like the combi function you placed. Both, combi and func2 work upto (10000,100) or (100000,80) for example. An n of n=10000 is a realistic number for my use, I think it won't be above that. However, I noticed that these huge numbers are being multiplied in the original code by really really small numbers (but not directly placable under the factorial product), so, now that I know the numerical limits of these combinatorials, I will try to multiply them by these small numbers before multiplying up to huge numbers (I hope to gain precision with that).
 
Some notes:
1. It's impossible to represent n! for n > 170 as a double value (170! ~ 7.25E306). All double n! values after 18! are factorial approximations only.
2. It's impossible to represent n! exactly for n > 20 as integer*8 value (20! = 2432902008176640000).
It's not Fortran issue: it's modern CPU's arithmetics.
Can't understand what else problems are you discussing here...
 
I will have to look for a reformulation of the problem from a math point of view first and then get back to fortran.
 
Status
Not open for further replies.

Part and Inventory Search

Sponsor

Back
Top