Hello and welcome to our community! Is this your first visit?
Register
Enjoy an ad free experience by logging in. Not a member yet? Register.
Results 1 to 2 of 2
  1. #1
    New Coder
    Join Date
    Oct 2016
    Posts
    36
    Thanks
    6
    Thanked 1 Time in 1 Post

    using the sigmoid-function to take large numbers and make them manegable in php

    Sometimes - if you have large databases/sites - you can end up dealing with large numbers to handle

    Let's say you are YouTube and need to order a serch reslut with 25,000,000 results and you want to order them by number of views and upvotes and... to get a relevance score - but then you have a couple of PewdePIe videos with 50,000,000 views and 2,000,000 upvotes and soon your numbers becomes a 'little big'
    ...

    luckily some smart people have decided to help you and have designed some mathematical functions to reduce those large numbers down to size

    one such function is the sigmoid function:
    S(x) = 1/(1+exp(-x))
    which greatly reduces the size of your numbers - but in php it only really works for numbers between about -700 and +32 - anything below will just return 0 and anything above will just return 1... so not much help after all

    so here's a little script that'll greatly increase the range of usable numbers by simply adding log10 to x in the expression the number range increases greatly - but now you can't 'srink' negative numbers

    well here is a little function to fix that (feel fre to use if needed)

    PHP Code:
    function sigmoid_log10($x) {
        if (
    $x<0) {
            return -(
    1/(1+exp(-log10(abs($x)))));
        } else {
            return 
    1/(1+exp(-log10($x)));
        }

    Now you can take any number between -10xE32 and +10xE32 - and with an acuracy of 10xE-32 reduce it to a usable value between -1 and +1

    And you can even use it in MYSQL directly:

    PHP Code:
    // use count(0) to get the values you want to use in your relevance-score
    SELECT 
        
    // select the values you need for your relevance score
        
    SELECT COUNT(0FROM videovote AS vvup WHERE vvup.videoID /* something */ AND vvup.upVote IS NOT NULL ) AS upvote,
        ( 
    SELECT COUNT(0FROM videovote AS vvdown WHERE vvdown.videoID /* something */ AND vvdown.downVote IS NOT NULL ) AS downvote,
        ( 
    SELECT COUNT(0FROM ... // you get the drill,
        
        // and then use the values you collected above and put it into the modified sigmoid function
        
    / ( EXP ( -LOG10 /* here goes your values to be scored for relevance */ ) ) ) ) AS score
        
        
    // and then just order by score
        
    ORDER BY score
        
        
    // example of a what's hot video score computation using this - but you can make your own collectiong the values you want and putting them into the formula above
        
    / ( EXP ( -LOG10 ( ( LOG10 ABS upvotedownvote) + ) * SIGN upvotedownvote) + LOG10 views ) + LOG10 commentsCount ) + ( UNIX_TIMESTAMP v.date ) / 600000 ) ) ) ) ) ) AS score 
    Feel free to use at your risk (as with great power comes great responsibility)
    Last edited by Thothlike; Dec 17th, 2017 at 03:33 PM. Reason: corrected [CODE] to [PHP]

  2. #2
    New Coder
    Join Date
    Oct 2016
    Posts
    36
    Thanks
    6
    Thanked 1 Time in 1 Post
    I just realized a glaring oversite in my MYSQL-code...

    as stated when it came to the PHP-function (sigmoid_log10) it only handles negative numbers with the addition of abs on the data and a minus in front for those negative numbers

    but I forgot to encorporate the same functionality into my MYSQL-code
    this is a huge oversite (and since the edit button is gone here is the updated MYSQL-code to incorporated neagative numbers as well)

    PHP Code:
    / ( EXP ( -LOG10 /* here goes your values to be scored for relevance */ ) ) ) ) AS score 
    should of course be
    PHP Code:
    ( ( / ( EXP ( -LOG10 ABS /* here goes your values to be scored for relevance */ ) ) ) ) ) * SIGN /* here goes the same values you just used to score for relevance */ ) ) AS score 
    adding the ABS to get the positive of whatever you are evaluating for you relevanse score and then multiplying the whole thing with SIGN of what you just used to evaluate (giving it a plus in front if your evaluation was positive and a minus in front if your evaluation was negative)

    and that offcourse also means that the 'what's hot' code
    PHP Code:
    / ( EXP ( -LOG10 ( ( LOG10 ABS upvotedownvote) + ) * SIGN upvotedownvote) + LOG10 views ) + LOG10 commentsCount ) + ( UNIX_TIMESTAMP v.date ) / 600000 ) ) ) ) ) ) AS score 
    using this to incoporate negative numbers should be
    PHP Code:
    / ( EXP ( -LOG10 ABS ( ( LOG10 ABS upvotedownvote) + ) * SIGN upvotedownvote) + LOG10 views ) + LOG10 commentsCount ) + ( UNIX_TIMESTAMP v.date ) / 600000 ) ) ) ) ) ) * SIGN ( ( LOG10 ABS upvotedownvote) + ) * SIGN upvotedownvote) + LOG10 views ) + LOG10 commentsCount ) + ( UNIX_TIMESTAMP v.date ) / 600000 ) ) ) ) AS score 


 

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •