datatables and Memcached, Redis

datatables and Memcached, Redis

javismilesjavismiles Posts: 205Questions: 38Answers: 3

Good day,
I use regularly Memcached to cache my database queries, great speed gains,
is it possible to connect memcached with datatables? and if so, any examples?
best

Answers

  • allanallan Posts: 61,446Questions: 1Answers: 10,054 Site admin

    Plain old DataTables doesn't really care where you get your data from, as long as it can read it (typically JSON). So if you can output JSON from your scripts, then yes, it would work just fine.

    Allan

  • javismilesjavismiles Posts: 205Questions: 38Answers: 3

    thank you Allan,
    yes I get that, but Im using the datatables PHP editor library, so I guess I would need to understand where I can plug memcached into there, because I know how to do it with my own php scripts, easy, but not sure where that would go within the editor php library, i would have to get inside the code, mmm

  • javismilesjavismiles Posts: 205Questions: 38Answers: 3

    if I understand well, this is where in the php editor library the json is obtained and sent back to javascript, somehow i would have to plugin though the memcached way before, before the whole process is executed,
    the key is here:

    $editor->process( $_POST );
    $editor->json();

    question, if I do

    $thejson= $editor->json();

    will I get in $thejson the json that is being sent back? because then I could put it then in memcache and then do

    ///// GET FROM MEMCACHE
    $memkey="getdb";
    if ($cacheAvailable == true) {
        $fromstore=$memcache->get($memkey); 
        if ($fromstore) {echo $fromstore;return;}
    }
    
       $editor->process( $_POST );
       $tosend=$editor->json();
    
       if ($cacheAvailable == true) {$memcache->set($memkey, $tosend,$cachedur);}
    
    
    
    
    
    
     */
    public function json ( $print=true )
    {
        if ( $print ) {
            $json = json_encode( $this->_out );
    
            if ( $json !== false ) {
                echo $json;
            }
            else {
                echo json_encode( array(
                    "error" => "JSON encoding error: ".json_last_error_msg()
                ) );
            }
    
            return $this;
        }
        return json_encode( $this->_out );
    }
    
  • javismilesjavismiles Posts: 205Questions: 38Answers: 3

    I Think I managed to install the memcache, I have to confirm that its working, but no errors, for those that may be interested in it, I put the code here, as this, if it works, its a huge benefit when used with any DB stuff:

    define('MEMCACHED_HOST', '127.0.0.1');
    define('MEMCACHED_PORT', '11211');
    $memcache = new Memcached;
    $cacheAvailable = $memcache->addServer(MEMCACHED_HOST, MEMCACHED_PORT);
    $cachedur=30;

    then at bottom of php:

    $memkey="db".$etype;
    if ($cacheAvailable == true) {
    $fromstore=$memcache->get($memkey);
    if ($fromstore) {echo $fromstore;return;}
    }

    $editor->process( $_POST );
    $tosend=$editor->json();

    and then inside Editor.php:

    public function json ( $print=true )
    {
    
        global $cacheAvailable;
        global $memcache;
        global $etype;
        global $cachedur;
    
        if ( $print ) {
            $json = json_encode( $this->_out );
    
            if ( $json !== false ) {
                echo $json;
            }
            else {
                echo json_encode( array(
                    "error" => "JSON encoding error: ".json_last_error_msg()
                ) );
            }
    
            return $this;
        }
    
        $memkey="db".$etype;
        $tosend=json_encode( $this->_out );
        if ($cacheAvailable == true) {$memcache->set($memkey, $tosend,$cachedur);}
    
        return json_encode( $this->_out );
    }
    
  • javismilesjavismiles Posts: 205Questions: 38Answers: 3

    im just not sure its working, i have to verify it

  • javismilesjavismiles Posts: 205Questions: 38Answers: 3

    I tested and so far its not working, no errors, but its not caching it, not sure why yet mmm

  • javismilesjavismiles Posts: 205Questions: 38Answers: 3

    I think im putting the code in the wrong place,
    I need guidance please as to which part of the php library is starting and finishing the process of gathering the db queries, thank u so much

  • javismilesjavismiles Posts: 205Questions: 38Answers: 3

    so it works now, it works,
    the key is that I was puttting the code in the wrong place of the json function, this position makes it work well:

    public function json ( $print=true )
    {

        global $cacheAvailable;
        global $memcache;
        global $etype;
        global $cachedur;
    
        if ( $print ) {
            $json = json_encode( $this->_out );
    
            if ( $json !== false ) {
    
    
                $memkey="db".$etype;
                $tosend=$json;
                if ($cacheAvailable == true) {$memcache->set($memkey, $tosend,$cachedur);}
    
    
                echo $json;
            }
            else {
                echo json_encode( array(
                    "error" => "JSON encoding error: ".json_last_error_msg()
                ) );
            }
    
            return $this;
        }
    
    
        return json_encode( $this->_out );
    }
    
  • allanallan Posts: 61,446Questions: 1Answers: 10,054 Site admin

    Thanks for posting back. Good to hear you've got it working now.

    Allan

  • javismilesjavismiles Posts: 205Questions: 38Answers: 3

    thank you Allan,
    yes it works but there is a "but",
    for this to work properly, the key has to take into account parameters such as:

    $start_p=$_POST['start'];
    $order_p1=$_POST['order'][0]['column'];
    $order_p2=$_POST['order'][0]['dir'];

    $memkey="db".$etype."-".$start_p."-".$order_p1."-".$order_p2;

    Applying this works so that you can still paginate and sort, however something weird happens, when changing the sorting of a column, after switching twice, the table stops being updated until the key expires. Looking at the network I can see the json being returned being apparently correct, there seems to be some weird interaction with the javascript and then it doesnt update until the key expires, its kind of weird

    $processnow=1;

    if ($cacheAvailable == true) {
    $fromstore=$memcache->get($memkey);
    if ($fromstore) {$processnow=0;
    echo $fromstore;
    return;}
    }

    if ($processnow==1){
    $editor->process( $_POST );
    $editor->json();
    }

    //////////////////////////////////////////////////////////////////////////////////// Editor.php

    public function json ( $print=true )
    {

        global $cacheAvailable;
        global $memcache;
        global $etype;
        global $cachedur;
        global $memkey;
    
        if ( $print ) {
            $json = json_encode( $this->_out );
    
            if ( $json !== false ) {
                $tosend=$json;
                if ($cacheAvailable == true) {$memcache->set($memkey, $tosend,$cachedur);}
    
                echo $json;
            }
    

    ..... etc

  • javismilesjavismiles Posts: 205Questions: 38Answers: 3

    yeah, ive tried with Redis also, same problem,
    it all works except that when coming and going between ASC and DESC of a sorting row or between pages of pagination area, for some reason, it wont update the second time the same key is requested before the key expires

  • javismilesjavismiles Posts: 205Questions: 38Answers: 3

    yes, a pity, printing the redis or memcached variables on separate pages I can see that all works perfect on the server side, no probs,
    for some reason the javascript doesnt wanna update the rows the second time it goes back to the same case

  • javismilesjavismiles Posts: 205Questions: 38Answers: 3

    I think i will try my best to see if I can provide you a private link with a test case for you to check if possible this issue and the one of the inline editing on compact mobile phone situation as I dont think i can fix this myself :)

  • javismilesjavismiles Posts: 205Questions: 38Answers: 3

    how can I send you a private link in a private message? thank you ;)

  • javismilesjavismiles Posts: 205Questions: 38Answers: 3

    memcache and redis when bserverside is true really dont work well, when bserverside is false then yes, it seems that it can work ok there, a pity though as I wanted it to work with bserverside true

  • allanallan Posts: 61,446Questions: 1Answers: 10,054 Site admin

    My guess is that the draw parameter is not being updated to match what DataTables sent from the client-side.

    To send a PM in the forum, click my user name above and then the Send message button.

    Allan

  • javismilesjavismiles Posts: 205Questions: 38Answers: 3

    thank you Allan, yes got a few things to show u, will prepare some test case soon and send you via PM, thank u again

This discussion has been closed.