Model Versioning
Over time, your model will change, and so its revision. Document then needs to be converted, and this chapter describes how to do so.
Converters
Your model
folder should contain a conv
folder. It will contains all your format version changes in a single place. Since Flip format conversion system is rather flexible, it is very easy to make document format changes, so you are very likely to make a lot of them.
You should name your conversion classes Converter#1_#2
where #1
would be the initial revision, according to your version source control naming. In the same fashion, #2
would be your destination revision.
Conversion graph
Just after reading a serialized document in BackEndMl
or BackEndBinary
to the BackEndIR
format, you need to convert the latter format to the revision of the model currently used in your application.
BackEndMl
or BackEndBinary
are model revision agnostic, and so is BackEndIR
, so that you don't need to maintain all versions of the model to be able to convert.
The general idea is to convert from one conversion to another the BackEndIR
format until you've matched the revision available to use with your application.
Typically, this global conversion process looks like this :
DataProviderFile provider ("/path/to/file"); |
// tell the backend to automatically recognize the binary format |
BackEndIR backend; |
backend.register_backend <BackEndBinary> (); |
bool ok_flag = backend.read (provider); |
if (!ok_flag) return; // a corruption occured |
// now starts the conversion process, where every converters process |
// the 'backend' in-place, and changes its version |
if (backend.version == "myapp.beta.1") |
{ |
Converter_beta1_beta2 converter; |
converter.process (backend); |
// now backend.version == "myapp.beta.2" |
} |
if (backend.version == "myapp.beta.3") |
{ |
// beta.3 was a mistake, so we decide to rollback it to beta.2 |
// for our loyal beta testers |
Converter_beta3_beta2 converter; |
converter.process (backend); |
// now backend.version == "myapp.beta.2" |
} |
if (backend.version == "myapp.beta.2") |
{ |
Converter_beta2_r100 converter; |
converter.process (backend); |
// now backend.version == "myapp.r100" |
// which is the current model revision |
} |
Document document (Model::use (), user_id, manufacturer_id, component_id); |
// change the document with the content of the backend |
document.read (backend); |
// commit the changes |
document.commit (); |
Converter Overview
Now that we have seen the global conversion graph managment, let's look at a single converter anatomy.
BackEndIR
contains a member version
string which is the current revision of the model. Apart from that, it contains a member root
which is the starting point of every conversions.
root
itself is a BackEndIR::Type
which is a generic way to represent a Type
in the document. It can represent either an Object
, but also a container type or a basic type like Float
.
Type
contains a type_name
as well as a type_id
to allows to know which kind of type we are manipulating.
- If the
Type
is an object, thenType::members
is the list of the members of the object - If the
Type
is an array, thenType::array
is the list of the element of the array - If the
Type
is a collection, thenType::collection
is the list of the element of the collection - If the
Type
is a map, thenType::map
is the list of the element of the map - If the
Type
is a value type likeFloat
, thenType::value
is the value
Important: Usually you will want to process the backend in-place. The function converting the backend shall be completely deterministic, and in particular should not be dependent of time or any source of random.
The rationale behind it is to make sure that any one on the network to the same document can convert the document to a new version and have exactly the same result without requiring any kind of locking.
Adding a Member to a Class
Let's consider the following model :
class Note : public Object |
{ |
public: |
String title; |
String body; |
}; |
class Root : public Object |
{ |
public: |
Collection <Note> notes; |
}; |
And that we want to convert to this new model, where we added a member date
class Note : public Object |
{ |
public: |
String title; |
String body; |
String date; |
}; |
class Root : public Object |
{ |
public: |
Collection <Note> notes; |
}; |
Members in an Object
in a BackEndIR
Type
are a list of pairs of a name and a Type
, where the name is the one as declared in the Flip class declaration.
WARNING: Your converter shall be deterministic and shall not be dependent on current date or time, or any source of randomness.
Then the converter will need to add a member and initialize it to a value this way :
void Converter_v100_v110::process (BackEndIR & backend) |
{ |
// !!! WARNING we must not use the current date WARNING !!! |
std::string default_date = "2017-01-12T13:26:07+00:00"; |
// fetch 'notes' member |
auto & notes_member = backend.root.member ("notes"); |
// member is a pair of name ("notes") and type (the collection itself) |
auto & notes = notes_member.second; |
for (auto & element : notes.collection) |
{ |
// 'element' is a pair of key and type |
auto & note = element.second; |
note.object_add_member_string ("date", default_date); |
} |
// when the conversion is done, bump the current backend version |
backend.complete_conversion ("v110"); |
} |
Renaming a Class
Let's consider the same starting previous model.
We now have two different classes for Note
s which derive from it, so that a Note
can either be a NoteText
or a NoteImage
. NoteText
will have the same structure as the previous Note
, but the class name change. Therefore we need to change every class that was named to Note
to NoteText
.
void Converter_v100_v110::process (BackEndIR & backend) |
{ |
auto & notes = backend.root.member ("notes").second; |
for (auto & element : notes.collection) |
{ |
auto & note = element.second; |
assert (note.get_class () == "Note"); |
note.object_set_class ("NoteText"); |
} |
backend.complete_conversion ("v110"); |
} |
Renaming a Class's member
Let's consider the same starting previous model.
Let's suppose we now want to rename the member body
to content
.
Members in an Object
in a BackEndIR
Type
are a list of pairs of a name and a Type
, where the name is the one as declared in the Flip class declaration.
void Converter_v100_v110::process (BackEndIR & backend) |
{ |
auto & notes = backend.root.member ("notes").second; |
for (auto & element : notes.collection) |
{ |
auto & note = element.second; |
note.object_rename_member ("body", "content"); |
} |
backend.complete_conversion ("v110"); |
} |
Adding a New Object to a Collection
Let's consider the same starting previous model.
Let's suppose we want to add a new note for the user to discover a new feature of our software. For that we are going to synthesize a new object and add it to our container.
Elements in a Collection
in a BackEndIR
Type
are a map of pairs of a key and a Type
.
When manipulating a Collection
in the model code, the key is just a random number. But because the converter processing function must be deterministic, we must use a pseudo source of randomness.
For this purpose, ConverterCollectionKeyGenerator
is used. When adding an element to the collection, the generator is used to generate numbers. The generator will be called as many time as needed until it produces a key that is not already into the Collection
.
Usually, one should use one generator for each converter, and seed it with a different number for each converter, mainly for performance reasons.
void Converter_v100_v110::process (BackEndIR & backend) |
{ |
ConverterCollectionKeyGenerator generator (backend); |
auto & notes = backend.root.member ("notes").second; |
auto & note = notes.collection_add_object ("Note", generator).second; |
note.object_add_member_string ("title", "New Note Filtering feature"); |
note.object_add_member_string ("body", "Filtering notes has never been easier!"); |
backend.complete_conversion ("v110"); |
} |
Adding a New Object to an Array
Let's consider the same starting previous model, with the only change that Note
s are now ordered with an Array
, so that Root
now looks like this :
class Root : public Object |
{ |
public: |
Array <Note> notes; |
}; |
Let's suppose we want to add a new note for the user to discover a new feature of our software. For that we are going to synthesize a new object and add it to our container.
Elements in an Array
in a BackEndIR
Type
are a map of pairs of a key and a Type
.
When manipulating an Array
in the model code, the key is a random floating number between two consecutive elements. But because the converter processing function must be deterministic, The key produced in the converter is just a fast computed center approximate of the two consecutive elements.
void Converter_v100_v110::process (BackEndIR & backend) |
{ |
auto & notes = backend.root.member ("notes").second; |
auto & note = notes.array_add_object ("Note", notes.array.end ()).second; |
note.object_add_member_string ("title", "New Note Filtering feature"); |
note.object_add_member_string ("body", "Filtering notes has never been easier!"); |
backend.complete_conversion ("v110"); |
} |
Adding a New Object to a Map
Let's consider the same previous model, where a Note
has 3 members title
, description
and date
.
class Note : public Object |
{ |
public: |
String title; |
String description; |
String date; |
}; |
This time we consider that we made a mistake into the original model, and that we want the concept of "dynamics object members".
Map
can be used to do that. Map
is very similar to the Object
members description, except that "members" are ordered by names. Namely, Map
is a map of pairs of a key and a Type
, where the key is a name. The name itself can be a number or a string.
class Property : public Object |
{ |
public: |
String value; |
}; |
class Note : public Object |
{ |
public: |
Map <Property> properties; |
}; |
In the following example, we move all 3 members into a Map
called properties
, and make the conversion accordingly. Because a Map
can only contain an Object
(as opposed to a basic type such as String
), we introduce a Flip class for that.
void Converter_v100_v110::process (BackEndIR & backend) |
{ |
auto & notes = backend.root.member ("notes").second; |
for (auto & element : notes.collection) |
{ |
auto & note = element.second; |
std::string title = note.member ("title").second; |
std::string description = note.member ("description").second; |
std::string date = note.member ("date").second; |
auto & properties = note.object_add_member_map ("properties").second; |
properties.map_add_object ("Property", "title") |
.object_add_member_string ("value", title); |
properties.map_add_object ("Property", "description") |
.object_add_member_string ("value", description); |
properties.map_add_object ("Property", "date") |
.object_add_member_string ("value", date); |
// remove old members |
note.object_remove_member ("title"); |
note.object_remove_member ("description"); |
note.object_remove_member ("date"); |
} |
backend.complete_conversion ("v110"); |
} |
Making a Modification Deep into the Data Model Tree
In the previous data model used, the deepness of the tree was small so that modifying objects was easy.
Now let's configer the following data model.
class Note : public Object |
{ |
public: |
String title; |
String body; |
}; |
class NoteBook : public Object |
{ |
public: |
Collection <Note> notes; |
}; |
class Root : public Object |
{ |
public: |
Collection <NoteBook> notebooks; |
}; |
Let's suppose we now want to rename the member body
of Note
to content
.
We can use the walk
function as a convenience to achieve that. The first parameter of the function is the BackEndIR
Type
to start from, and the second function to execute on every Type
objects.
In the following example, we apply walk
on the document root so that we process the whole document.
void Converter_v100_v110::process (BackEndIR & backend) |
{ |
walk (backend.root, [](BackEndIR::Type & type){ |
// skip every type that is not a "Note" |
if (type.get_class () != "Note") return; |
auto & member = type.member ("body"); |
member.first = "content"; |
}); |
backend.complete_conversion ("v110"); |
} |
Moving an Object within the Data Model Tree
Let's consider the same starting model. We now are considering to introduce the concept of NoteBook
s and to move all Notes
to a default NoteBook
.
class Note : public Object |
{ |
public: |
String title; |
String body; |
}; |
class NoteBook : public Object |
{ |
public: |
String title; |
Collection <Note> notes; |
}; |
class Root : public Object |
{ |
public: |
Collection <NoteBook> notebooks; |
}; |
To do so, we will :
- Create one
Notebook
- Copy every
Note
into theNotebook
Note
- Remove every previous
Note
void Converter_v100_v110::process (BackEndIR & backend) |
{ |
auto & notebooks = backend.root.object_add_member_collection ("notebooks").second; |
auto & notebook = notebooks.collection_add_object ("NoteBook").second; |
notebook.object_add_member_string ("title", "My First Notebook"); |
auto & dst_notes = notebook.object_add_member_collection ("notes"); |
auto & src_notes = backend.root.member ("notes").second; |
for (auto & element : src_notes.collection) |
{ |
// add the element as a pair (key/value) |
dst_notes.collection_add (element); |
} |
backend.root.object_remove_member ("notes"); |
backend.complete_conversion ("v110"); |
} |
Handling Conversion Errors
Whenever an unexpected error is triggered, such as trying to access a non-existant member or trying to insert an element which key already exist in a container, the functions described above will throw
using flip_THROW
which in turn throws a std::runtime_error
exception.
The user of the framework should also complies to this scheme for the unexpected errors that might be triggered while converting.
Since the whole conversion system is user side code, exception handling shall be provided by the user of the framework.
If the backend is corrupted but could be successfully translated, then one of the following errors can happen :
Document::read
will throw because it can't apply the documentDocument::commit
will throw because the change fails validation
bool load (Document & document, const char * path_0) |
{ |
DataProviderFile provider (path_0); |
BackEndIR backend; |
backend.register_backend <BackEndBinary> (); |
bool ok_flag = backend.read (provider); |
if (!ok_flag) |
{ |
// the file format is corrupted or the file couldn't be read |
return false; |
} |
// call the conversion graph |
try |
{ |
convert (backend); |
} |
catch (...) |
{ |
// conversion failed, backend is corrupted |
return false; |
} |
// change the document with the content of the backend |
try |
{ |
document.read (backend); |
} |
catch (...) |
{ |
// backend is corrupted |
return false; |
} |
// commit the changes |
try |
{ |
document.commit (); |
} |
catch (...) |
{ |
// validation failed |
return false; |
} |
return true; |
} |
See Also
The present guide describes only the most important and basic scheme of document conversion.
More details are available in BackEndIR::Type reference documentation.