Communities

Writing
Writing
Codidact Meta
Codidact Meta
The Great Outdoors
The Great Outdoors
Photography & Video
Photography & Video
Scientific Speculation
Scientific Speculation
Cooking
Cooking
Electrical Engineering
Electrical Engineering
Judaism
Judaism
Languages & Linguistics
Languages & Linguistics
Software Development
Software Development
Mathematics
Mathematics
Christianity
Christianity
Code Golf
Code Golf
Music
Music
Physics
Physics
Linux Systems
Linux Systems
Power Users
Power Users
Tabletop RPGs
Tabletop RPGs
Community Proposals
Community Proposals
tag:snake search within a tag
answers:0 unanswered questions
user:xxxx search by author id
score:0.5 posts with 0.5+ score
"snake oil" exact phrase
votes:4 posts with 4+ votes
created:<1w created < 1 week ago
post_type:xxxx type of post
Search help
Notifications
Mark all as read See all your notifications »
Q&A

Comments on How to determine a precise upper bound of the growth rate with a product series?

Parent

How to determine a precise upper bound of the growth rate with a product series?

+4
−0

I'm trying to determine the growth rate of an algorithm.

The variables are:

n: the input size; it's an integer >=1.
p: a coefficient in the range (0, 1).

So far, I've been able to determine a growth rate upper bound in terms of a product:

product(log(n*p^k) / log(1/(1-p))),
for k from 0 to (log(n)/log(1/p));
$$ \Pi_{k = 0}^{\log n / \log(1/p)} \log(np^k)/\log(1/(1-p)) $$

How can I transform that series into big-O notation?

Of course, I can just consider an upper bound by taking k==0 in the whole series, and say it's in

O((log(n) / log(1/(1-p))) ^ (log(n) / log(1/p) + 1));

But is there a more precise upper bound thanks to the increasing value of k?


I've been able to extract the denominator, as it doesn't depend on k.

product(log(n *    p^k) / log(1/(1-p))),
 for k from 0 to (log(n)/log(1/p));
=
product(log(n *    p^k))
/ log(1/(1-p)) ^ (log(n)/log(1/p) + 1),
 for k from 0 to (log(n)/log(1/p));

And now, I can separate the logarithms into a sum, to extract the k from the exponent (I'm not sure if it helps).

product(log(n *    p^k))
/ log(1/(1-p)) ^ (log(n)/log(1/p) + 1),
 for k from 0 to (log(n)/log(1/p));
=
product(log(n)+log(p)*k)
/ log(1/(1-p)) ^ (log(n)/log(1/p) + 1),
 for k from 0 to (log(n)/log(1/p));

And now it looks like (a+b)^x, but not exactly. log(n) is always log(n). But log(p)*k is increasing in magnitude (and <0).

We can invert the log to get a negative sign, which makes it more normalized:

product(log(n) + k*log(p))
/ log(1/(1-p)) ^ (log(n)/log(1/p) + 1),
 for k from 0 to (log(n)/log(1/p));
=
product(log(n) - k*log(1/p))
/ log(1/(1-p)) ^ (log(n)/log(1/p) + 1),
 for k from 0 to (log(n)/log(1/p));

And it is interesting to see that in the last iteration where k=log(n)/log(1/p), the multiplicand is log(n) - log(n)/log(1/p)*log(1/p), which is equal to 1. The multiplicand is at a maximum in the first iteration, and then decays to 1. Is that decay powerful enough to counter the number of elements in the series?

History

0 comment threads

Post
+2
−0

I suspect there's a factorial here. Ok, let's try to isolate the k.

product(log(n) - k*log(1/p))
/ log(1/(1-p)) ^ (log(n)/log(1/p) + 1)),
 for k from 0 to (log(n)/log(1/p));
=
product(log(1/p)*(log(n)/log(1/p) - k))
/ log(1/(1-p)) ^ (log(n)/log(1/p) + 1)),
 for k from 0 to (log(n)/log(1/p));

And then we can move out that factor from the product, as it doesn't depend on k.

product(log(1/p)*(log(n)/log(1/p) - k))
/ log(1/(1-p)) ^ (log(n)/log(1/p) + 1),
 for k from 0 to (log(n)/log(1/p));
=
product(log(n)/log(1/p) - k)
* log(1/p)     ^ (log(n)/log(1/p) + 1)
/ log(1/(1-p)) ^ (log(n)/log(1/p) + 1),
 for k from 0 to (log(n)/log(1/p));
=
product(log(n)/log(1/p) - k)
* (log(1/p)/log(1/1-p)) ^ (log(n)/log(1/p) + 1),
 for k from 0 to (log(n)/log(1/p));

And now we can see it nicely that there's a factorial.

product(log(n)/log(1/p) - k)
* (log(1/p)/log(1/1-p)) ^ (log(n)/log(1/p) + 1),
 for k from 0 to (log(n)/log(1/p));
=
(log(n)/log(1/p))!
* (log(1/p)/log(1/1-p)) ^ (log(n)/log(1/p) + 1)

And here it is. We got rid of the product, and have it as a factorial (which is still a product, but one more well known).

And let's try to move that log(1/p) to the denominator, to normalize it further:

(log(n)/log(1/p))! * (log(1/p)/log(1/1-p)) ^ (log(n)/log(1/p) + 1)
=
(log(n)/log(1/p))! / (log(1/1-p)/log(1/p)) ^ (log(n)/log(1/p) + 1)

(I might have made some mistakes, though; the graph is similar to what I get experimentally, but has some important differences.)

History

1 comment thread

It looks right to me; once you get to \[\product{k=0}^{\frac{\log(n)}{\log\left(\frac1p\right)}} (\lo... (9 comments)
It looks right to me; once you get to \[\product{k=0}^{\frac{\log(n)}{\log\left(\frac1p\right)}} (\lo...
clemens‭ wrote 6 months ago · edited 6 months ago

It looks right to me; once you get to $\Pi_{k=0}^{\frac{\log(n)}{\log\left(\frac1p\right)}} (\log(n) - k·\log(\frac1p))$ at the beginning of your post it's actually more than clear that that grows at the rate of the factorial. (I didn't see it earlier only because I wasn't paying attention to the range over which the product was being taken.)

Of course once we get a factorial we can just use Stirling's approximation, if so desired, to get a more tractable form of our approximation.

alx‭ wrote 6 months ago · edited 6 months ago

Hmmm, thanks! Let's try Stirling's approximation.

(log(n)/log(1/p))!
/ (log(1/1-p)/log(1/p)) ^ (log(n)/log(1/p) + 1)
~=
sqrt(2*pi* log(n)/log(1/p))
* ((log(n)/log(1/p)) /e) ^ (log(n)/log(1/p))
/ (log(1/1-p)/log(1/p)) ^ (log(n)/log(1/p) + 1)
=
sqrt(2*pi* log(n)/log(1/p))
* ((log(n)/log(1/p)) /e) ^ (log(n)/log(1/p) + 1)
/  (log(n)/log(1/p)) /e
/ (log(1/1-p)/log(1/p)) ^ (log(n)/log(1/p) + 1)
=
sqrt(2*pi* log(n)/log(1/p))
* (log(n) /log(1/p)^2 /log(1/1-p) /e) ^ (log(n)/log(1/p) + 1)
/ (log(n) /log(1/p)) /e
=
sqrt(2*pi* log(n)/log(1/p))
* (log(n) /log(1/p)^2 /log(1/1-p) /e) ^ (log(n)/log(1/p) + 1)
/ (log(n) /log(1/p))
/ e
=
sqrt(2*pi)/e
* (log(n) /log(1/p)^2 /log(1/1-p) /e) ^ (log(n)/log(1/p) + 1)
/ sqrt(log(n)/log(1/p))

Where the dominant part is the middle one.

(This thing shows a significantly different graph in WolframAlpha for n=10^3. I might have goofed it.)

alx‭ wrote 6 months ago

clemens‭ Is this transformation correct?

alx‭ wrote 6 months ago · edited 6 months ago

clemens‭

BTW, am I right thinking this is sub-exponential (but no polynomial) on n (considering the worst case of p, which is moving near 0.9 for large n)?

Considering p more or less fixed, this seems to be in O(log(n) ^ log(n)), right? (With rather large constants, though.)

Should I maybe ask a separate question for this?

clemens‭ wrote 6 months ago

Yes, it probably ought to go into a separate question. But how do you get the transition from the third to the fourth formula in your comment above? (Maybe I'm missing it...)

alx‭ wrote 6 months ago · edited 6 months ago

clemens‭

Oh, I had a mistake there. (I accidentally did ^2 where I should have cancelled out two elements). I'll simplify after the 3rd formula (hopefully, this time without mistakes):

sqrt(2*pi* log(n)/log(1/p))
* ((log(n)/log(1/p)) /e) ^ (log(n)/log(1/p) + 1)
/  (log(n)/log(1/p)) /e
/ (log(1/1-p)/log(1/p)) ^ (log(n)/log(1/p) + 1)
=
sqrt(2*pi* log(n)/log(1/p))
* ((log(n)/log(1/p)) /e) ^ (log(n)/log(1/p) + 1)
/ (log(1/1-p)/log(1/p)) ^ (log(n)/log(1/p) + 1)
/  (log(n)/log(1/p)) /e
=
sqrt(2*pi* log(n)/log(1/p))
* (((log(n)/log(1/p)) /e) / (log(1/1-p)/log(1/p))) ^ (log(n)/log(1/p) + 1)
/  (log(n)/log(1/p)) /e
=
sqrt(2*pi* log(n)/log(1/p))
* (((log(n)/       1) /e) / (log(1/1-p)/       1)) ^ (log(n)/log(1/p) + 1)
/  (log(n)/log(1/p)) /e
=
sqrt(2*pi* log(n)/log(1/p))
* (((log(n)/log(1/1-p)) /e)) ^ (log(n)/log(1/p) + 1)
/  (log(n)/log(1/p)) /e
=
sqrt(2*pi/e)
* sqrt(log(n)/log(1/p))
* (((log(n)/log(1/1-p)) /e)) ^ (log(n)/log(1/p) + 1)
/  (log(n)/log(1/p))

...

alx‭ wrote 6 months ago · edited 6 months ago

...

sqrt(2*pi/e)
* sqrt(log(n)/log(1/p))
* (((log(n)/log(1/1-p)) /e)) ^ (log(n)/log(1/p) + 1)
/  (log(n)/log(1/p))
=
sqrt(2*pi/e)
* (  log(n)/log(1/1-p)  /e ) ^ (log(n)/log(1/p) + 1)
/ sqrt(log(n)/log(1/p))
clemens‭ wrote 6 months ago · edited 6 months ago

This is correct. Do the asymptotics still appear wrong?

alx‭ wrote 6 months ago

clemens‭

The asymptotics seem correct now. Thanks!

I'll ask the separate question about the big-O of this.