Hello and welcome to our community! Is this your first visit?
Enjoy an ad free experience by logging in. Not a member yet? Register.

# Thread: using the sigmoid-function to take large numbers and make them manegable in php

1. ## using the sigmoid-function to take large numbers and make them manegable in php

Sometimes - if you have large databases/sites - you can end up dealing with large numbers to handle

Let's say you are YouTube and need to order a serch reslut with 25,000,000 results and you want to order them by number of views and upvotes and... to get a relevance score - but then you have a couple of PewdePIe videos with 50,000,000 views and 2,000,000 upvotes and soon your numbers becomes a 'little big'
...

luckily some smart people have decided to help you and have designed some mathematical functions to reduce those large numbers down to size

one such function is the sigmoid function:
S(x) = 1/(1+exp(-x))
which greatly reduces the size of your numbers - but in php it only really works for numbers between about -700 and +32 - anything below will just return 0 and anything above will just return 1... so not much help after all

so here's a little script that'll greatly increase the range of usable numbers by simply adding log10 to x in the expression the number range increases greatly - but now you can't 'srink' negative numbers

well here is a little function to fix that (feel fre to use if needed)

PHP Code:
``` function sigmoid_log10(\$x) {    if (\$x<0) {        return -(1/(1+exp(-log10(abs(\$x)))));    } else {        return 1/(1+exp(-log10(\$x)));    }}  ```
Now you can take any number between -10xE32 and +10xE32 - and with an acuracy of 10xE-32 reduce it to a usable value between -1 and +1

And you can even use it in MYSQL directly:

PHP Code:
``` // use count(0) to get the values you want to use in your relevance-scoreSELECT     // select the values you need for your relevance score    ( SELECT COUNT(0) FROM videovote AS vvup WHERE vvup.videoID = /* something */ AND vvup.upVote IS NOT NULL ) AS upvote,    ( SELECT COUNT(0) FROM videovote AS vvdown WHERE vvdown.videoID = /* something */ AND vvdown.downVote IS NOT NULL ) AS downvote,    ( SELECT COUNT(0) FROM ... // you get the drill,        // and then use the values you collected above and put it into the modified sigmoid function    ( 1 / ( 1 + EXP ( -LOG10 ( /* here goes your values to be scored for relevance */ ) ) ) ) AS score        // and then just order by score    ORDER BY score        // example of a what's hot video score computation using this - but you can make your own collectiong the values you want and putting them into the formula above    ( 1 / ( 1 + EXP ( -LOG10 ( ( LOG10 ( ABS ( upvote- downvote) + 1 ) * SIGN ( upvote- downvote) + LOG10 ( views + 1 ) + LOG10 ( commentsCount + 1 ) + ( UNIX_TIMESTAMP ( v.date ) / 600000 ) ) ) ) ) ) AS score  ```
Feel free to use at your risk (as with great power comes great responsibility)

2. I just realized a glaring oversite in my MYSQL-code...

as stated when it came to the PHP-function (sigmoid_log10) it only handles negative numbers with the addition of abs on the data and a minus in front for those negative numbers

but I forgot to encorporate the same functionality into my MYSQL-code
this is a huge oversite (and since the edit button is gone here is the updated MYSQL-code to incorporated neagative numbers as well)

PHP Code:
``` ( 1 / ( 1 + EXP ( -LOG10 ( /* here goes your values to be scored for relevance */ ) ) ) ) AS score  ```
should of course be
PHP Code:
``` ( ( 1 / ( 1 + EXP ( -LOG10 ( ABS ( /* here goes your values to be scored for relevance */ ) ) ) ) ) * SIGN ( /* here goes the same values you just used to score for relevance */ ) ) AS score  ```
adding the ABS to get the positive of whatever you are evaluating for you relevanse score and then multiplying the whole thing with SIGN of what you just used to evaluate (giving it a plus in front if your evaluation was positive and a minus in front if your evaluation was negative)

and that offcourse also means that the 'what's hot' code
PHP Code:
``` ( 1 / ( 1 + EXP ( -LOG10 ( ( LOG10 ( ABS ( upvote- downvote) + 1 ) * SIGN ( upvote- downvote) + LOG10 ( views + 1 ) + LOG10 ( commentsCount + 1 ) + ( UNIX_TIMESTAMP ( v.date ) / 600000 ) ) ) ) ) ) AS score  ```
using this to incoporate negative numbers should be
PHP Code:
``` ( 1 / ( 1 + EXP ( -LOG10 ( ABS ( ( LOG10 ( ABS ( upvote- downvote) + 1 ) * SIGN ( upvote- downvote) + LOG10 ( views + 1 ) + LOG10 ( commentsCount + 1 ) + ( UNIX_TIMESTAMP ( v.date ) / 600000 ) ) ) ) ) ) * SIGN ( ( LOG10 ( ABS ( upvote- downvote) + 1 ) * SIGN ( upvote- downvote) + LOG10 ( views + 1 ) + LOG10 ( commentsCount + 1 ) + ( UNIX_TIMESTAMP ( v.date ) / 600000 ) ) ) ) AS score  ```

#### Posting Permissions

• You may not post new threads
• You may not post replies
• You may not post attachments
• You may not edit your posts
•