Flip Basics Programming Guide

Working with a Remote Server

At this point, you are able to declare a model and interact with it by controlling it and observing it. But still this happens only on a local computer.

This chapter exposes how Flip works with a remote server and brings collaboration features to your application.

Commit/Push/Pull Model

When you call commit on a document, Flip will create the transaction that represents the change between the old and new states of the document.

Additionnaly, Flip will store this transaction in a Push Stack ready to be pushed to the remote server.

If you then call push on the document, and the document is set up to work with a remote server, Flip will send all the commits to that server, so that they can be executed by the server and arbitrated.

Similarly, to receive the changes available on the remote server done by another user, or to know if the change on a document were acknowledge or denied, one will call pull on the document to receive and merge this changes into your local document. Internally, the transport layer will regurarly fetch the server for changes and store them in a Pull Stack. Calling pull will simply execute each element of the pull stack.

Generally, you won't push to the remote server for every single commit you do. For example in the previous example, if a user drags a clip on the track by changing its position, then you may want to advertise those changes to the outside world only when the user has finished its gesture by releasing the mouse button. In this case you might do commit for every mouse moves, and then only push to the remote server once the user has released the mouse button. Depending on the case, you might also want not to commit on every mouse moves, and commit and push at the same time when the user released the mouse button.

Additionally, one can squash the Push Stack, so that all transactions previously commited are squashed to one transaction before this transaction is pushed to the server. This is particularly useful in the case where the same values are changed over and over in a gesture, so that the resulting transaction is far smaller than every commits.

Note: Because the Push Stack is only used when the document is connected to a carrier, squashing will return an empty transaction if the document is not connected to a carrier

Generally, you would pull almost constantly from the server.

The next sections will show how to set up concretely the code to work with a remote server.

Working with a Remote Server, Client Part

Documents connect to a remote a server through a Transport. Flip can support many transports methods through its transport abstraction.

For a simple TCP socket, one can use CarrierTransportSocketTcp in flip/contrib/transport_tcp. This will be the class used in the following examples.

A client is assigned a user identifier. This user identifier must be unique for every user connecting to the same session. Typically this can be a facebook user number or an OpenId unique identifier. For a quick test, using a random generator might be good enough.

Then the client is going to connect to a session, with a session identifier. A session represents a document on the server side, and is usually meant to be persistent.

#include "flip/contrib/transport_tcp/CarrierTransportSocketTcp.h"
void  run (uint64_t user_id, uint64_t session_id)
{
   Document document (Model::use (), user_id, 'appl', 'gui ');                   (1)
   CarrierTransportSocketTcp carrier (document, session_id, "localhost", 9000);  (2)
   carrier.process ();                                                           (3)
   document.pull ();                                                             (4)
   Root & root = document.root <Root> ();
   root.happiness_level = 100.0;
   document.commit ();
   document.push ();                                                             (5)
   carrier.process ();
}
  1. Create a new document for user_id. user_id must be unique for every user connecting to the session
  2. Connect to localhost on port 9000, for session_id
  3. process allows for this transport implementation to write and read from the socket as needed
  4. this call to pull will replace the current document with the one from the remote server
  5. this call to push will send the happiness level change to the document server

Note: The code above is simplified, as a single process code on a transport carrier does not guarantee that the socket will be able to send or receive all data.

The code above implies that a Flip server is running on localhost port 9000. The next section will show how to set up this server.

Working with a Remote document, Server Part

In this example, ServerSimple from flip/contrib is used. It is a simple server that allows to host multiple sessions, and is running all in one process. This is probably not suitable for production quality, but nethertheless very handy to set up a flip server quickly.

#include "flip/contrib/DataConsumerFile.h"
#include "flip/contrib/DataProviderFile.h"
#include "flip/contrib/RunLoopTimer.h"
#include "flip/contrib/ServerSimple.h"
#include "flip/BackEndBinary.h"
std::string path (uint64_t session_id)
{
   return std::string ("/sessions/") + std::to_string (session_id);
}
void run ()
{
   ServerSimple server (Model::use (), 9000);               (1)
   server.bind_validator_factory ([](uint64_t session_id){
      return std::make_unique <MyModelValidator> ();        (2)
   });
   server.bind_init ([](uint64_t session_id, DocumentBase & document){
      Root & root = document.root <Root> ();
      root.happiness_level = 42.0;                          (3)
   });
   server.bind_read ([](uint64_t session_id){
      BackEndIR backend;
      auto session_path = path (session_id);
      if (path_exists (session_path))
      {
         DataProviderFile provider (session_path.c_str ());
         backend.register_backend <BackEndBinary> ();
         backend.read (provider);
      }
      return backend;                                       (4)
   });
   server.bind_write ([](uint64_t session_id, const BackEndIR & backend){
      auto session_path = path (session_id);
      DataConsumerFile consumer (session_path.c_str ());
      backend.write <BackEndBinary> (consumer);             (5)
   });
   RunLoopTimer run_loop ([&server](){
      server.process ();
      return true;
   });
   run_loop.run ();                                           (6)
}
  1. Set up the server to listen on port 9000
  2. Bind the validator factory. This is run every time a new session is opened
  3. Bind the document initiator. When no session is already present (see 4. below), it will create a non-empty initial document, should it be needed.
  4. Bind the document reader. Here it is using a DataProviderFile with the BackEndBinary production document format. If the returned backend is empty, then the document initiator (see 3. above) is called. This is also where you would convert your document to the current model version if needed.
  5. Bind the document writer. Here it is using a DataConsumerFile with the BackEndBinary production document format
  6. Finally run the server. This function will allow the server to process incoming client request

When a user connects to a session, a validator is created for this session if the client of the framework provided a validator factory function through bind_validator_factory.

Then the optionally provided read function through bind_read is called. If this function is not provided or it returns an empty backend, then the optionally provided init function through bind_init is called. In particular the init function is not called if the read function returns a non-empty backend.

Finally the write function provided through bind_write is called when every users of a session leave, just before destroying the session.

It is also possible to customize your own run loop. This allows for example to run other services, monitor the server sessions, or make regular snapshots of the sessions.

void  my_runloop_callback (ServerSimple & server, bool snapshot_flag)
{
   server.process ();                              (1)
   if (snapshot_flag)
   {
      for (auto && session : server.sessions ())
      {
         session.write ();                         (2)
      }
   }
}
void run ()
{
   [...]
   RunLoopTimer run_loop ([&server](){
      snapshot_flag = need_snapshot (std::chrono::system_clock::now ());
      my_runloop_callback (server, snapshot_flag);
      return true;                                 (3)
   });
   run_loop.run ();
}
  1. process needs to be called regularly
  2. Explicitly flush the session, this will call the bound write method provided earlier
  3. Return true if the run loop should still run or false to exit the run loop

Validating Transactions

When a single user is working in a document, it can somehow be assumed that the changes made for the document are going to be valid, from the model logic point of view. While this can be ensured by proper coding in the single user case, there is no way to make sure of it in a concurrent environment.

To solve this problems, Servers need to run validation code. The code will ensure that the new state of the document is valid from the model logic point of view. If the document is not valid, the validator will inform it to the Flip system which will rollback the transaction, ensuring that the document returns to the last known valid state.

Validating the model is very similar to observing it. Actually the code to traverse the model will be exactly similar to the one to observe it. When the validator encounters an error, it will throw to notify the Flip system that the document state it is looking at is not valid. The convenience macro flip_VALIDATION_FAILED is provided for this matter.

Declaring and Binding a Validator

Validators are declared in a similar way to observers :

#include "flip/DocumentValidator.h"
class Validator : public flip::DocumentValidator <Song>                       (1)
{
public:
   virtual void validate (Song & song) override;                              (2)
}
void  run ()
{
   ServerSimple server (Model::use (), 9000);
   server.bind_validator_factory ([](uint64_t session_id){
      return std::make_unique <Validator> ();                                 (3)
   });
}
  1. Make a validator by inheriting from DocumentValidator
  2. Override the pure virtual function declared in DocumentValidator
  3. Create a new Validator that will be attached to every new document created by the server

Now, every time the document session will be changed and the server receives this modification, validate will be fired.

Writing Validation Code

In the example, we might want to ensure that the clip positions are always positive or zero, and that the durations are always strictly positive. Implementing this validation scheme would be done like in the following code :

void  Validator::validate (Song & song)
{
   if (song.tempo.changed ())
   {
      if (song.tempo < 20) flip_VALIDATION_FAILED ("incorrect tempo");
   }
   if (song.tracks.changed ())
   {
      for (auto && track : song.tracks)
      {
         if (track.changed ())
         {
            for (auto && clip : track.clips)
            {
               if (clip.changed ())
               {
                  if (clip.position < 0.0) flip_VALIDATION_FAILED ("incorrect position");
                  if (clip.duration <= 0.0) flip_VALIDATION_FAILED ("incorrect duration");
               }
            }
         }
      }
   }
}

The next chapter, More Fun with Flip will guide you through more advanced use of Flip.