Config Packages

The goal of the conffwk package is to provide user-friendly API to access data from the configuration database. There are two layers of such API which can be seen by user:

• abstract conffwk layer working with arbitrary database schema and hiding details of DBMS implementation
• data access library (DAL), that is generated for given database schema to map it on programming language data types

This page describes basics which a user should know to generate DAL from the database schema, to get data from database, to receive notification on their change, to create new data or to modify existing data using generated DAL.

• 1. Development of the configuration database schema
• 2. Generation of the DAL
  • 2.1. Parameters of genconffwk utility
  • 2.2. Integration with CMake
• 3. DAL classes and methods
  • 3.1. Mapping Between OKS Attribute Types and Programming Languages Types
• 4. Errors Handling
• 5. How to get data
  • 5.1. Initialization
  • 5.2. Read objects of class
  • 5.3. Reading Values of Attributes
  • 5.4. Reading Values of Relationships
  • 5.5. Cast Class Types
  • 5.6. Data Destruction
• 6. How to create and to modify data
  • 6.1. Creation of new database file
  • 6.2. Database Includes
  • 6.3. Objects Manipulations
  • 6.4. Modification Values of Attributes
  • 6.5. Modification Values of Relationships
  • 6.6. Modification of Database and Invalid Objects
• 7. Notification mechanism
  • 7.1. User Callback
  • 7.2. Subscription criteria
  • 7.3. Subscription
• 8. Algorithms
• 9. Advanced Code Generation

Sections 1, 2 and 3 are needed for those, who develops own schema and wants to generate DAL. Section 4 explain basics of error handling to be known by any user of DAL. Section 5 explain how to get data using DAL. Section 6 explains how to create or to modify data using DAL. Section 7 explains how to receive notification in case of data changes. The possibility to plug-in user algorithms to the generated DAL is described in section 8. Section 9 explains how to extend generated classes inserting user code.

1. Development of the configuration database schema

It is possible to develop a new schema in case one needs to describe own configuration data which cannot be described by existing schemes. The development of the schema can be done using the OKS Schema Editor. There is a choice to extend the existing schemes or to develop own schema from scratch. To extend existing schema it is necessary to start the editor with it (which cannot be modified) and then to create the schema that can be updated, e.g.:

oks_schema_editor dal/schema/core.schema.xml

The editor window with loaded schema will appear. Then it is possible to create own schema and to define own classes. To create a new schema from scratch, one just needs to run the editor without parameters and to create a new schema. For more information on the OKS schema editor, the OKS schema capabilities and exporting schema into different formats see OKS documentation. After the schema development is finished, it is necessary to save the schema into xml file and add it to the sources of some package. Such schema file will be used for the database data access library generation described by next section.

2. Generation of the DAL

A DAL is generated by the genconffwk utility. It uses OKS schema files as input and produces:

• C++ source files to build the library
• C++ header files to describe library interface
• C++ files to build binaries dumping content of the database
• Java files to build jar file
• genconffwk.info file containing information about names of generated classes, the C++ code namespace, include prefix directory and java package name

2.1. Parameters of genconffwk utility

The command line parameters of genconffwk utility are listed below:

genconffwk [-d | --C++-dir-name directory-name]
          [-n | --C++-namespace namespace
          [-i | --C++-headers-dir directory-prefix]
          [-j | --java-dir-name directory-name]
          [-p | --java-package-name package-name]
          [-I | --include-dirs dirs*]
          [-c | --classes class*]
          [-D | --user-defined-classes [namespace::]user-class[@dir-pefix]*]
          [-f | --info-file-name file-name]
          [-v | --verbose]
          [-h | --help]
          -s | --schema-files file.schema.xml+
 
Options/Arguments:
    -d directory-name    name of directory for C++ header and implementation files
    -n namespace         namespace for C++ classes
    -i directory-prefix  name of directory prefix for C++ header files
    -j directory-name    name of directory for java files
    -p package-name      package name for java files
    -I dirs*             directories where to search for already generated files
    -c class*            explicit list of classes to be generated
    -D [x::]c[@d]*       user-defined classes
    -f filename          name of output file describing generated files
    -v                   switch on verbose output
    -h                   this message
    -s files+            the schema files (at least one is mandatory)

To generate a DAL it is necessary to provide name of the schema file. By default the DAL is generated for all classes contained in the schema files, otherwise one should provide names of required classes via –classes parameter. It is recommended to use unique namespace for each generated DAL to avoid possible problems when several DALs are used by one application.

It is possible to reuse already generated DALs. In this case one should to provide a list of directories containing information about already generated DALs via --include-dirs parameter, then to specify list of schema files and optionally to specify list of names of classes to be generated. It is expected such schema files use include statement for base schema files. The DAL is generated only for classes contained in the explicitly mentioned schema files, the classes from included files are ignored.

2.2. Integration with CMake

See CMake for ATLAS TDAQ Software TWiki page

The tdaq_generate_dal() command should be used to generate C++ and Java code from OKS schema files. The full signature looks like this:

tdaq_generate_dal(schema_file...
  NAMESPACE ns
  [INCLUDE_DIRECTORIES ...]
  [CLASSES ...]
  [PACKAGE name]
  [INCLUDE dir]
  [NOINSTALL]
  [CPP dir] [JAVA dir]
  [CPP_OUTPUT var1] [JAVA_OUTPUT jvar2]
  [DUMP_OUTPUT var2])

The command will generate the C++ and/or Java code and put the names of the generated files in the variables passed in the CPP_OUTPUT and JAVA_OUTPUT arguments respectively. Here is a simple use case:

tdaq_generate_dal(data/schema/myclasses.schema.xml
  NAMESPACE MyNamespace
  INCLUDE_DIRECTORIES dal
  INCLUDE DFConfiguration
  CPP_OUTPUT dal_cpp_srcs)
 
  tdaq_add_library(mylib DAL ${dal_cpp_srcs})

This generates the C++ code for the classes in the schema file, using the namespace MyNamespace and referring to previously declared classes from the dal package. The header files will be installed into include/DFConfiguration. Note that the user doesn't have to know the names of the generated files. They are simply passed on to the library that uses them. In a similar way, the Java sources can be generated (typically using the PACKAGE argument to specify the generated package name. You can also generate the C++ and Java with a single command. The Java sources should be passed to the tdaq_add_jar() command.

If you want to build the generated dump command for this schema, specify the DUMP_OUTPUT variable which will contain the source file for the dump command. You can pass that to tdaq_add_executable().

For both C++ and Java based libraries the DAL option should be specified when generated source files are used. This ensures that certain race conditions during the generation and the usage of the files are properly handled. The most common use case of having a single DAL and one or more libraries/jars using it is covered in this case.

If you are in the unusual situation that you generate more than one DAL in the same package, you have to do a bit more work: First pass the TARGET target_name parameter to tdaq_generate_dal(). It will uses this name instead of a default to represent the generation of the source files. Then use the add_dependencies() command to explicitly add dependencies between the DAL generation target and any consumers of the generated files. The internal target is the name you give, prefixes with DAL_.

tdaq_generate_dal(TARGET dal1 schema/s1.schema.xml ...CPP_OUTPUT dal1_sources)
tdaq_generate_dal(TARGET dal2 schema/s2.schema.xml ...CPP_OUTPUT dal2_sources)
 
tdaq_add_library(consumer1 ${dal1_sources})
add_dependencies(consumer1 DAL_dal1)
 
tdaq_add_library(consumer2 ${dal2_sources})
add_dependencies(consumer2 DAL_dal2)

You may not encounter any problems during development at all, especially if you don't do a highly parallel build. However, during the nightly builds we use all the cores of the machines we are running on, and if there are users in other packages of your library, the probability of race conditions increases.

3. DAL classes and methods

For each OKS class appropriate DAL classes are generated:

• in case of C++ the generated class has the same name as OKS one and is declared inside namespace defined by the user; there is separate header file per each class; it has the same name as the database class and, to be included, it may have directory prefix, defined by user; if a class is derived from other classes, an appropriate C++ inheritance is used;
• in case of Java there is interface which has the same name as the database class declared inside package with name provided by the user; the interface implementation is in the class with suffix _Impl; the static methods to get existent or to create new objects of the class are in the class with suffix _Helper; an appropriate inheritance is used between interfaces.

For each direct attribute and relationship defined for OKS class the appropriate methods are generated. Such methods have the same names as the names of the attributes and the relationships in the database with get_ and set_ prefixes. The database attribute types are mapped to appropriate C++ and Java types. The multi-value attributes are mapped to std::vector of attribute type in C++ and to array of attribute type in Java. The database relationships are mapped to methods returning pointer or std::vector of pointers to objects of referenced class in C++ and similarly an object or array of objects in Java.

Additionally, for each class there are methods to get object's class name and object identity as they are defined in the database.

For C++ two std::ostream operators are generated for each class:

• the one with const reference to object prints out the full description of the object, and
• the other one with const pointer to object prints out the object's class name and identity.

In Java for each class there is generated method print() which prints out full description of the object.

When DAL is generated, any non-alphanumeric characters appeared in the names of classes, attributes, relationships and methods are replaced by underscore symbol, e.g. database attribute "# of c++ lines" will appear in DAL as "__of_c___lines".

3.1. Mapping Between OKS Attribute Types and Programming Languages Types

Below there is map between OKS attribute types and C++/Java types:

OKS Type	C++ type	Java type
bool	bool	boolean
s8 (8-bits signed integer)	int8_t	char
u8 (8-bits unsigned integer)	uint8_t	byte
s16 (16-bits signed integer)	int16_t	short
u16 (16-bits unsigned integer)	uint16_t	short
s32 (32-bits signed integer)	int32_t	int
u32 (32-bits unsigned integer)	uint32_t	int
s64 (64-bits signed integer)	int64_t	long
u64 (64-bits unsigned integer)	uint64_t	long
float	float	float
double	double	double
date	std::string	java.lang.String
time	std::string	java.lang.String
string	std::string	java.lang.String
enum	std::string	java.lang.String
class	std::string	java.lang.String

In C++ the complex values (strings and vectors) are passed by const reference and not by value.s

4. Errors Handling

Methods of C++ and Java conffwk classes throw exceptions in case of errors.

C++ methods either have explicit noexcept specification if they may not throw an exception, or declare exception specification in doxygen documentation. The following exceptions can be thrown:

• daq::conffwk::Generic is used to report most of the problems (bad DB, wrong parameter, plug-in specific, etc.)
• daq::conffwk::NotFound the conffwk object accessed by ID is not found, class accessed by name is not found
• daq::conffwk::DeletedObject accessing template object that has been deleted (via notification or by the user's code)

All above exceptions have common class daq::conffwk::Exception, that can be used to catch all of them.

try {
    // load database using oks file /tmp/mydb.data.xml
  Configuration db("oksconflibs:/tmp/mydb.data.xml");
 
  ... // user's code working with db
}
catch (daq::conffwk::Exception & ex) {
    // throw some user-defined exception in case of conffwk exception
  throw ers::error(user::exception(ERS_HERE, "cannot read conffwk", ex));
}

5. How to get data

The entry point to get data from database is the class Configuration defined in global namespace in C++ and in the conffwk package in Java. Below it is described how to create an object of this class and how to use it to get the database information.

5.1. Initialization

The Configuration constructors in both Java and C++ languages have single string parameter. If the parameter is an empty string, the constructor will use value of the TDAQ_DB environment variable. In case if both the constructor parameter and the environment variable are not set, or empty, then the configuration constructor throws daq::conffwk::Generic in case of C++, or conffwk.SystemException is thrown in case of Java.

The format of the parameter is "name-of-plugin:plugin-parameters". The plug-in's parameter is optional. If it is non-empty, it is passed to the implementation plug-in constructor.

In C++ the name of plug-in is converted into name of the shared library by adding prefix lib and suffix .so, e.g. "oksconflibs" plug-in name is converted into "liboksconflibs.so". The shared library must be in the path to shared libraries, e.g. in the LD_LIBRARY_PATH environment variable.

In Java the name of plug-in is converted into name of the class in package "plugin-name" with name created from "plugin-name", where 1-st and 4-th characters are converted to upper case and "uration" string is appended (it is so by historical reasons), e.g. "oksconflibs" plug-in name is converted into "oksconflibs.OksConfiguration". The CLASSPATH variable has to point to such class or jar file.

For the moment three implementation plug-ins are available:

• the oksconflibs using OKS implementation directly (i.e. reads XML files),
• the read-only roksconflibs using OKS archive (oks2coral) (i.e. reads OKS data from Oracle archive),
• the rdbconffwk accessing OKS with RDB server.

Details of initialization in C++ and examples

Below there are examples of the Configuration constructor explicit parameters:

#include "conffwk/Configuration.h"
 
try {
    // example (1): load daq/partitions/be_test.data.xml file using oks
  ::Configuration db1("oksconflibs:daq/partitions/be_test.data.xml");
 
    // example (2): connect with server RDB using rdb implementation (in initial partition)
  ::Configuration db2("rdbconffwk:RDB");
 
    // example (2a): connect with server RDB using rdb implementation (in test partition)
  ::Configuration db2a("rdbconffwk:test::RDB");
 
    // example (2b): same as above using new style server-name@partition-name
  ::Configuration db2b("rdbconffwk:RDB@test");
 
    // example (3): use oks implementation and create new database
  ::Configuration db3("oksconflibs");
  db3.create("", "/tmp/my.data.xml", std::list<std::string>(1,"/tmp/my.sch.xml"));
 
    // example (4): use rdb implementation and create new database
    //              on server RDB running in partition test
  ::Configuration db4("rdbconffwk");
  db4.create("test::RDB", "/tmp/my.data.xml", std::list<std::string>());
}
catch(daq::conffwk::Exception& ex) {
  std::cerr << "ERROR: " << ex << std::endl;
}

The recommended way is to get plug-in and it's parameter via environment variable. Most of the user's code for applications run by TDAQ's setup should to leave the parameter empty:

#include "conffwk/Configuration.h"
 
int main() {
  try {
    ::Configuration db("");
    ERS_DEBUG( 1 , "Read database " << db.get_impl_spec())
    db.get(...) // any user's code working with Configuration object
  }
  catch(daq::conffwk::Exception & ex) {
    std::cerr << "ERROR: " << ex << std::endl; return 1;
  }
}

In case it the parameter can be also passed via command line, use it as shown below:

#include "conffwk/Configuration.h"
 
int main(int argc, char *argv[]) {
  try {
    ::Configuration db(argv[1]);
    ERS_DEBUG( 1 , "Read database " << db.get_impl_spec())
    db.get(...) // any user's code working with Configuration object
  }
  catch(daq::conffwk::Exception & ex) {
    std::cerr << "ERROR: " << ex << std::endl; return 1;
  }
}

Note, to get proper plug-in name and parameter used for configuration initialization (e.g. obtained via TDAQ_DB), use get_impl_spec() method of Configuration class, which is used in above examples for debug reporting.

Initialization in Java

The Configuration constructor parameters are the same, as in case of C++. In case of problems the exception is thrown. An example is shown below:

import conffwk.Configuration;
 
try {
  conffwk.Configuration db = new conffwk.Configuration("rdbconffwk:RDB");
}
catch (conffwk.SystemException ex) {
  System.err.println( "ERROR caught \'conffwk.SystemException\':");
  System.err.println( "*** " + ex.getMessage() + " ***" );
}

Note in case when rdb implementation is used, the partition name of the RDB server can be specified by several ways:

• using the same approach as for C++, i.e. via constructor parameter using double colon-separated partition and server names, e.g. "rdbconffwk:partition-name::server-server" or "rdbconffwk:server-server@partition-name";
• via tdaq.ipc.partition.name java virtual machine property, e.g. run java application with "-Dtdaq.ipc.partition.name=partition-name".

5.2. Read objects of class

Once an object of the Configuration class is successfully created, it can be used to get configuration data (i.e. objects). Normally to get configuration objects only C++ template methods of the Configuration class and generated Java code should be used. The usage of conffwk layer (i.e. direct usage of objects of ConfigObject class) only makes sense in few packages working with arbitrary database schemes.

For each generated class T two methods can be applied using configuration object:

• C++ template methods of the ::Configuration class:
  • const T * Configuration::get(const std::string&, bool, bool, unsigned long, const std::vector<std::string> *) - to read named object;
  • void Configuration::get(std::vector<const T*>&, bool, bool, const std::string&, unsigned long rlevel, const std::vector<std::string> *) - to read objects of class;
• Java methods generated in class T_Helper:
  • static public T get(conffwk.Configuration db, String id) to read named object
  • static public T[] gets(conffwk.Configuration db, Query query) to read objects of class

The methods looking for single object by identity and methods looking for objects of class in case if the query string is empty are searching objects in the class and all it's subclasses, e.g. if class B is derived from class A, then the second method used for class A returns all objects of class A and all objects of class B, and the first method used for class A is looking for object with given identity in class A and then in class B. The same methods used for class B are only looking for objects of class B.

If the query is non-empty, the methods filling vectors of objects only return objects satisfying the query criteria. A query can be an OKS query string. It can be created by the OKS Data Editor or written by hand as described by the OKS documentation , e.g.:

• (all ("Name" "my-object" =)) - search all objects of class T and it's subclasses which name is equal to "my-object";
• (this (and ("Address" 128 >=) ("Address" 256 <))) - search all objects of class T which address is equal or greater than 128 and less than 256;
• (this ("Modules" all ("State" 0 =))) - search all objects of class T which has objects referenced via relationship "Modules" with attribute "State" set to 0.

C++ Example (using online dal package)

To read all applications via C++ dal provided by the online software one can write:

#include <conffwk/ConfigObject.h>
#include <conffwk/Configuration.h>
#include <dal/Application.h>
 
try {
  ::Configuration db("");
    // 'objects' vector contains Applications and all objects from derived classes,
    // e.g. RunControlApplications, etc.
  std::vector<const dal::Application *> objects;
  db.get(objects);
}
catch(daq::conffwk::Exception & ex) {
  std::cerr << "ERROR: " << ex << std::endl; return 1;
}

To get only run control applications it is possible to use the following code:

#include <dal/RunControlApplication.h>
try {
  ::Configuration db("");
  std::vector<const dal::RunControlApplication *> objects;
  db.get(objects);
}
catch(daq::conffwk::Exception & ex) {
  std::cerr << "ERROR: " << ex << std::endl; return 1;
}

Note, to get a named object it is necessary to use template parameter explicitly and to write:

const dal::Application * a = db.get<dal::Application>("my-application");

instead of

const dal::Application * a = db.get("my-application"); // COMPILATION ERROR!

When the methods parameter init_children is set to true, all objects referenced by the retrieved objects are also read and initialized (i.e. values of their attributes are read from database and all referenced objects are also recursively read). Otherwise the referenced objects can only be pre-allocated (if they were not already read explicitly) and the actual reading will happen, when the user will apply a method to read values of their attributes or relationships.

When the methods parameter init_object is set to false, all retrieved objects are only pre-allocated without reading their attributes and relationships. The values of attributes and relationships will actually be read from database implementation, when the user will apply a method to read an attribute or relationship value.

The above two parameters can be used by the user to improve performance. For example, if the parameter init_children is set to false, the only objects which are really used by the user process will be read from the database. However in this case the database can not be closed (e.g. to free used resources) until the configuration data are used. Also, the actual read from database can happen at an unexpected moment, that can introduce undesired delays.

Java Example (online DAL)

To read all applications via Java dal provided by the online software one can write:

import conffwk.Configuration;
import dal.Application;
import dal.Application_Helper;
...
conffwk.Configuration db = new conffwk.Configuration(...);
try {
  dal.Application objs[] = dal.Application_Helper.get(db, new conffwk.Query());
} catch ( conffwk.NotFoundException ex ) {
  System.err.println("ERROR: bad query or no such class loaded");
} catch ( conffwk.SystemException ex ) {
  System.err.println("ERROR: caught system exception");
}

To get only run control applications it is possible to write (try/catch statements are skipped):

import dal.RunControlApplication;
import dal.RunControlApplication_Helper;
...
dal.RunControlApplication objs[] = dal.RunControlApplication_Helper.get(db, new conffwk.Query());

Below there is an example of code to get an application by ID. Note, if object with such ID does not exist, conffwk.NotFoundException exception is thrown.

try {
  dal.Application obj = dal.Application_Helper.get(db, "RootController");
}
catch (conffwk.NotFoundException e) {
  System.err.println( "ERROR: can not find application \'RootController\'" );
}

5.3. Reading Values of Attributes

Once the objects are retrieved, the user can get values of their attributes. A method to read attribute value is created for each attribute of each generated class. It has the following format:

• for C++:
  • type get_AttributeName() const - for single-value integer and float numbers;
  • const std::string& get_AttributeName() const - for single-value string-based attributes;
  • const std::vector<type>& get_AttributeName() const - for multi-value attributes;
• for Java:
  • type get_AttributeName() const - for single-value attributes;
  • type[] get_AttributeName() const - for multi-value attributes.

Attribute Converters

The user can use one or several ways to convert values of all attributes of a C++ or Java type. To do this he/she needs to implement or to use already existing converter class, to create converter object of that class and to pass such object to the Configuration object using method Configuration::register_converter().

In case of C++ such class has to inherit from the template Configuration::AttributeConverter < T > class, where template parameter T defines type of attributes which values need to be converted and to implement virtual method Configuration::AttributeConverter::convert(), that performs the real conversion of attribute values.

Note

The C++ converter object is destroyed by the configuration destructor. User must not call delete on the attribute converter object.

In case of Java a converter class has to implement conffwk.AttributeConverter interface defining two methods: the method convert() as in C++ and the method get_class(), which returns class of converted attributes.

Below there is C++ example of user functions converting string and integer attributes:

#include <conffwk/Configuration.h>
 
  // converter to replace by '_' a non-alpha-numeric symbol in db strings
class GoodString : public ::Configuration::AttributeConverter<std::string> {
public:
  static char cvt_symbol(char c) { return (isalnum(c) ? c : '_'); }
  void convert(std::string& s, const Configuration&, const ConfigObject&, const std::string&) {
    std::transform(s.begin(), s.end(), s.begin(), cvt_symbol);
  }
}
  // converter to make any long integer value positive
class PositiveInt : public ::Configuration::AttributeConverter<unsigned long> {
  void convert(unsigned long& i, const Configuration&, const ConfigObject&, const std::string&) {
    if(i < 0) i = -i;
  }
}
...
try {
  ::Configuration db("");
  db.register_converter(new GoodString());
  db.register_converter(new PositiveInt());
  ...
}
catch(daq::conffwk::Exception & ex) {
  std::cerr << "ERROR: " << ex << std::endl; return 1;
}

In the Java an example of Java code is shown below:

import conffwk.Configuration;
import conffwk.AttributeConverter;
 
  // converter removes leading and trailing whitespace from a string
public class TrimString implements conffwk.AttributeConverter {
  public Object convert(Object s, conffwk.Configuration db, conffwk.ConfigObject obj, String attr_name) { return (Object)(s.trim()); }
  public Class get_class() { return String.class; }
}
...
conffwk.Configuration db = new conffwk.Configuration("");
db.register_converter(new TrimString());

Online DAL Converters

The core TDAQ C++ DAL (libdaq-core-dal.so) provides converter daq::core::SubstituteVariables class to substitute configuration parameters in values of string attributes. It's constructor requires Configuration object and Partition object, since they are used to calculate conversion map. In case, if configuration database is reloaded, such parameters have to be reset using reset() method. An example of the C++ code is shown below:

#include <conffwk/Configuration.h>
#include "dal/Partition.h"
#include <dal/util.h>
 
try {
  ::Configuration db("");
  if(const daq::core::Partition * p = daq::core::get_partition(db, "partition-X")) {
    db.register_converter(new daq::core::SubstituteVariables(db, *p));
  }
}
catch(daq::conffwk::Exception & ex) {
  std::cerr << "ERROR: " << ex << std::endl; return 1;
}

Similar Java dal.jar provides attribute converter in the same way. An example of it's usage is shown below:

import conffwk.Configuration;
import conffwk.DalObject;
import dal.SubstituteVariables;
import dal.Partition;
 
conffwk.Configuration db = new conffwk.Configuration("");
dal.Partition p = dal.Algorithms.get_partition(db, "partition-X");
if(p != null) {
  db.register_converter(new dal.SubstituteVariables(db, p));
}

5.4. Reading Values of Relationships

Once an object is retrieved, the user can get objects referenced by it. A method is created for each relationship of each generated class. It has the following format:

• for C++:
  • const class-type * get_RelationshipName() const - for single-value relationships;
  • const std::vector<const class-type*>& get_RelationshipName() const - for multi-value relationships;
• for Java:
  • class-type get_RelationshipName() const - for single-value relationships;
  • class-type[] get_RelationshipName() const - for multi-value relationships.

5.5. Cast Class Types

There are situations when user may need to cast an object from one class to a derived one. To make a down cast for an object of generated class the user should to use the methods of the configuration classes and never use cast supported by the programming languages.

C++ cast

There are situations when some set of objects can belong to different classes, e.g. objects can be of class A or B which is derived from class A. For a down cast the ::Configuration::cast() method must be used. As an example, the code to try and to cast from application to run-control application type is shown below:

try {
    // some code to get the vector of applications
  const std::vector<const dal::Application*>& l = ...;
  for(auto& j : l) {
    if(const dal::RunControlApplication * r = j->cast<dal::RunControlApplication>()) {
      std::cout << "application " << r << " is run control application" << std::endl; 
    }
  }
}
catch(daq::conffwk::Exception & ex) {
  std::cerr << "ERROR: " << ex << std::endl; return 1;
}

Java cast

To down cast an object of generated Java DAL use appropriate cast() method in generated class. For example, some object of application class can be down casted to the run control application:

dal.Application a = ...; // some code to get application
dal.RunControlApplication rc_application = dal.RunControlApplication_Helper.cast(a);
if(rc_application != null) { ... }

5.6. Data Destruction

In C++ the object of the ::Configuration class should not be destroyed while the DAL is in use. All objects read via template methods are destroyed by the ::Configuration class destructor. The user must never try to modify or to destroy such objects himself.

6. How to create and to modify data

This section explains how to create a new database file, how to create or remove database data and how to modify existing data.

Any modifications described by this section becomes persistent and visible to others processes only after successful commit operation. If the modification should not be committed (e.g. a modification failed), it is necessary to execute abort operation, e.g. in C++:

try {
  ::Configuration db();
  bool success = true;
  ... // some code which makes changes and sets the variable to false if failed
  if(success) {
    std::cout << "commit changes\n";
    db.commit();  // one also check return status, true means success
  }
  else {
    std::cerr << "ERROR: something was wrong, roll back changes\n";
    db.abort();
  }
}
catch(daq::conffwk::Exception & ex) {
  std::cerr << "ERROR: " << ex << std::endl; return 1;
}

Below there is the same example for Java:

conffwk.Configuration db = new conffwk.Configuration(...);
... // some code to makes changes and sets the success variable to false if failed
if(success) {
  System.out.println("commit changes");
  db.commit();
}
else {
  System.err.println("ERROR: something was wrong, roll back changes");
  db.abort();
}

To modify or to destroy an object using generated C++ DAL methods described below, it is necessary to have a non-const pointer or reference on the object. However all generated DAL methods return objects as const. To make a change it is necessary to use C++ const_cast to get non-const pointer or reference.

6.1. Creation of new database file

To create a new database file using C++ it is necessary to build an object of the ::Configuration class only providing name of implementation plug-in:

::Configuration db("oksconflibs");

Similar code for Java is below:

conffwk.Configuration db = new conffwk.Configuration("rdbconffwk"); // no db file

To create a new database data file it is necessary to decide which schema (at least one schema is always required) and optionally others database files will be used. Then it is necessary to provide an absolute name for newly created database file (the user should have write permission or the rdb server must be run in read-write mode under account which has such rights). If rdb implementation is used, it is also necessary to provide server and optionally partition name. After this it is necessary to use create method of the ::Configuration class and check it's return status.

Below there is example for C++ and oks implementation:

try {
  ::Configuration db("oksconflibs");
  std::list<std::string> includes;
  includes.push_back("online/schema/online.schema.xml"); // common schema
  includes.push_back("online/segments/setup.data.xml");  // online infrastructure
  const char * db_name = "/tmp/my-partition.data.xml";   // new database file name
  if(db.create("", db_name, includes) == false) {
    std::cerr << "ERROR: failed to create file " << db_name << std::endl;
    db.abort();
  }
}
catch(daq::conffwk::Exception & ex) {
  std::cerr << "ERROR: " << ex << std::endl; return 1;
}

For rdb implementation it is similar, but requires rdb server and optionally partition's names:

try {
  ::Configuration db("rdbconffwk");
  std::list<std::string> includes;
  includes.push_back("online/schema/online.schema.xml"); // common schema
  includes.push_back("online/segments/setup.data.xml");  // online infrastructure
  const char * db_name = "/tmp/my-partition.data.xml";   // new database file name
  const char * server_name = "foo::bar";                 // server with name bar running in part. foo
  if(db.create(server_name, db_name, includes) == false) {
    std::cerr << "ERROR: failed to create file " << db_name << " on " << server_name << std::endl;
    db.abort();
  }
}
catch(daq::conffwk::Exception & ex) {
  std::cerr << "ERROR: " << ex << std::endl; return 1;
}

For Java an example with rdb implementation is shown below:

try {
  conffwk.Configuration db = new conffwk.Configuration("rdbconffwk");
  String[] includes = new String[2];
  includes[0] = "online/schema/online.schema.xml"; // common schema
  includes[1] = "online/segments/setup.data.xml";  // online infrastructure
  db.create("foo::bar", "/tmp/my-partition.data.xml", includes);
  db.commit();
}
catch(conffwk.SystemException ex) {
  System.err.println("ERROR: caught \'conffwk.System\' exception");
}
catch(conffwk.NotAllowedException ex) {
  System.err.println("ERROR: caught \'conffwk.NotAllowed\' exception");
}
... // catch conffwk.AlreadyExistsException in a similar way

The included files should exist in advance and be defined either as an absolute path or as a relative path to a token of the TDAQ_DB_PATH variable value.

6.2. Database Includes

There are methods in C++ class ::Configuration to add a new include, to remove an existing include or to get list of includes for given database. They are:

• bool Configuration::add_include(const std::string& db_name, const std::string& include) - adds include to the database db_name and returns true in case of success or false if failed;
• bool Configuration::remove_include(const std::string& db_name, const std::string& include) - removes an existing include from the database db_name and returns true in case of success or false if failed;
• bool Configuration::get_includes(const std::string& db_name, std::list<std::string>& includes) const - fills list of includes by files which are included by the db_name and returns true in case of success or false if failed.

Similar methods in Java class conffwk.Configuration are:

• void add_include(String db_name, String include) - adds include to the database db_name or throws exception if failed;
• void remove_include(String db_name, String include) - removes an existing include from the database db_name or throws exception if failed;
• void get_includes(String db_name, String[] includes) - fills array of includes by files which are included by the db_name or throws exception if failed.

6.3. Objects Manipulations

This subsection explains how to create and how to destroy database objects.

Objects Creation

To create a new object using generated C++ DAL there are two ::Configuration template methods:

• const T * Configuration::create(const std::string& at, const std::string& id, bool) - to create new object of class T with identity id at existing database file with name at;
• const T * Configuration::create(const ::DalObject& at, const std::string& id, bool) - to create new object of class T with identity id at a database file where object at is stored.

The methods return non-null pointer in case of success or null if failed. The second method is faster since time to search the database file where to put new object is much smaller.

When the init_object parameter is set to false, then the values of attributes and relationships are not read from implementation (for a newly created object they are set to default values in accordance with the database schema).

An example how to create two new objects of the online dal::Computer class is shown below:

try {
  ::Configuration db(...);
  const char * dbfile = "/tmp/my-db.data.xml";
  const dal::Computer * host = db.create<dal::Computer>(dbfile, "host-1");
  if(host == 0) {
    std::cerr << "ERROR: failed to create object \'host-1\' at \'" << dbfile << "\'\n";
  }
  else {
    if(db.create<dal::Computer>(*host, "host-2") == 0) {
      std::cerr << "ERROR: failed to create object \'host-2\' at file of " << host;
    }
  }
}
catch(daq::conffwk::Exception & ex) {
  std::cerr << "ERROR: " << ex << std::endl; return 1;
}

On Java similar methods are genereted in the helper classes. For class T two methods are available:

• T create(conffwk.Configuration db, String at, String id) - to create new object of class T with identity id at existing database file with name at;
• T create(conffwk.Configuration db, conffwk.DalObject at, String id) - to create new object of class T with identity id at a database file where object at is stored.

The example to create online segment and it's application is shown below::

conffwk.Configuration db = new ...
String db_file = "/tmp/my-db.data.xml";
try {
  dal.Segment s = dal.Segment_Helper.create(db, db_file, "my segment");
  dal.Application a = dal.Application_Helper.create(db, s, "my application");
}
catch(conffwk.SystemException ex) {
  System.err.println("ERROR: caught \'conffwk.System\' exception");
} ... // also other exceptions to be caught

Objects Destruction

To destroy an existing object there is template method in the C++ ::Configuration class bool destroy(T& obj). It returns true in case of success and false if failed. See example:

try {
  ::Configuration db(...);
  dal::Computer * host = ...; // some code to get pointer
  db.destroy(*host);
}
catch(daq::conffwk::Exception & ex) {
  std::cerr << "ERROR: " << ex << std::endl; return 1;
}

In Java the method void destroy(conffwk.Configuration db) is generated in T.java, e.g.:

conffwk.Configuration db = new ...;
dal.Computer host = ...; // some code to get object
try {
  host.destroy(db);
}
catch(conffwk.SystemException ex) {
  System.err.println("ERROR: failed to destroy " + host);
} ... // also other exceptions to be caught

6.4. Modification Values of Attributes

Once the objects are retrieved or created, the user can modify values of their attributes. A method to set attribute value is created for each attribute of each generated class. The mapping between C++/Java types and OKS types can be seen in the 3.1. Mapping Between OKS Attribute Types and Programming Languages Types section.

In C++ such method throws daq::conffwk::Exception if failed:

• void set_AttributeName(type value) - for single-value attribute;
• void set_AttributeName(const std::vector<type>& value) - for multi-value attribute.

In Java such method throws an exception if failed:

• void set_AttributeName(type value) - for single-value attribute;
• void set_AttributeName(type[] value) - for multi-value attribute.

6.5. Modification Values of Relationships

Once the objects are retrieved or created, the user can modify values of their relationships. A method to set relationship value is created for each relationship of each generated class.

For C++ it has the following format and throws daq::conffwk::Exception if failed:

• void set_RelationshipName(const class-type * value) - for single-value relationships;
• void set_RelationshipName(const std::vector<const class-type*>& value) - for multi-value relationships.

For Java it has the following format and throws an exception if failed:

• void set_RelationshipName(class-type value) - for single-value relationships;
• void set_RelationshipName(class-type[] value) - for multi-value relationships.

6.6. Modification of Database and Invalid Objects

There are several methods which may make instances of ConfigObject and generated DAL classes be invalid:

• Configuration::remove_include(const std::string&, const std::string&) - this method destroys objects belonging to files closed in result of include removal;
• Configuration::destroy_obj(ConfigObject&) and template Configuration::destroy(T&) - those methods destroy given object and may destroy other objects linked via composite dependent relationships.

In case of ConfigObject there is no simple way to know if an object is valid after above remove_include() or destroy_obj() method call, since by efficiency reasons all methods are redirected to implementation object without checking of it's validity. It is recommended to reinitialize all instances of ConfigObject after above calls.

With objects of generated DAL classes the situation is different. After implementation object destruction a method invoked on corresponding DAL objects will throw daq::conffwk::DeletedObject exception, that can be caught by user code. Also, validity of object can be tested with DalObject::was_removed() method.

An example is shown below for include file removal and exception mechanism for generated DAL objects:

Configuration db;
 
...
 
std::vector<const daq::core::Computer*> nodes;
db.get(nodes);
 
  // remove include "bar" from file "foo",
  // that may result some files to be closed
  // and several template objects to be invalidated
 
db.remove_include("foo", "bar");
 
for(auto& i : nodes) {
  try {
    i->get_State();  // cause exception if object was destroyed
    std::cout << "object " << i << " was not destroyed" << std::endl;
  }
  catch(const daq::conffwk::DeletedObject&) {
    std::cout << "Oops, object " << (void *)i << " was destroyed" << std::endl;
  }
}

7. Notification mechanism

The user application can be notified on changes of the configuration data. To do this user should to implement one or many callback functions (C++) or classes (Java) which will be used when the database changes are committed and to choose which changes in classes and objects should be reported (i.e. to define the subscription criteria ).

The user receives description of information changes in one go via callbacks invoked after commit of database changes. This is more preferred way than individual callback per object or per class since user may want to see all changes at single point. Each callback receives own list of changes in accordance with it's subscription criteria.

The changes are reported as a collection of changes per DAL class. A change per class contains 4 parameters: the class name and the identities of created, modified and removed objects.

To make a subscription it is necessary to make three steps:

• implement callback,
• define subscription criteria,
• invoke subscribe method with above entities on the configuration object.

7.1. User Callback

To start with any subscription on database changes the user must to implement at least one ::Configuration::notify callback function in C++ or conffwk.Callback interface on Java. Below there are details for C++ and Java subscriptions.

C++ callback function

The user has to implement ::Configuration::notify callback. It has the following parameters:

• const std::vector<::ConfigurationChange *> & changes - description of changes
• void * parameter - user parameter

The ::ConfigurationChange class is declared in the conffwk/Change.h file and has 4 methods to get name of the class and vectors of created, modified and removed object identities. An example of callback functions is shown below:

void callback(const std::vector< ConfigurationChange *> & changes, void *)
{
  std::cout << "The CALLBACK reports all changes:\n";
 
  // iterate changes sorted by classes
  for(auto& j : changes) {
 
    // print class name
    std::cout << "- there are changes in class \"" << j->get_class_name() << "\"\n";
 
    // print modified objects
    for(auto& i : j->get_modified_objs()) {
      std::cout << "  * object \""  << i << "\" was modified\n";
    }
 
    // print removed objects
    for(auto& i : j->get_removed_objs()) {
      std::cout << "  * object \""  << i << "\" was removed\n";
    }
 
    // print created objects
    for(auto& i : j->get_created_objs()) {
      std::cout << "  * object \""  << i << "\" was created\n";
    }
  }
}

Java callback interface

The user has to create a class implementing the conffwk.Callback interface. It requires to implement method void process_changes(conffwk.Change[] changes, java.lang.Object parameter). Example below illustrates how to implement notification callback:

class TestCallback implements conffwk.Callback {
  private conffwk.Configuration db;
 
  public TestCallback(conffwk.Configuration d) { db = d; }
 
  public void process_changes(conffwk.Change[] changes, java.lang.Object parameter) {
    
      // the parameter can be any; as an example, the callback ID is passed as string
    String cb_id = (String)parameter;
 
      // print out changes description
    System.out.println("[TestCallback " + cb_id + "] got changes:");
    conffwk.Change.print(changes, "  ");
 
      // iterate changes by classes
    for(int i = 0; i < changes.length; i++) {
      conffwk.Change change = changes[i];
      System.out.println("* there are changes in the \'" + change.get_class_name() + "\' class");
 
        // just as example, look for changed objects of the Application class
      if((change.get_class_name().equals("Application") == true) && (change.get_changed_objects() != null)) {
        System.out.println("* " + change.get_changed_objects().length + " updated objects of the Application class");
 
          // iterate by all changed objects and print them out
    for(int j = 0; j < change.get_changed_objects().length; ++j) {
      dal.Application a = dal.Application_Helper.get(db, change.get_changed_objects()[j]);
 
        // an example of correct down cast
      if(a.class_name().equals("RunControlApplication")) {
        dal.RunControlApplication_Helper.get(db, a.config_object()).print("  "); // print as RC application
      }
      else {
        a.print("  "); // print as an application
      }
        }
      }
    }
  }
}

7.2. Subscription criteria

The subscription criteria is an object of ::ConfigurationSubscriptionCriteria class in C++ or conffwk.Subscription class in Java. It is used to define lists of classes and objects, which changes will be monitored and reposted to user. If user provides no any class or object, it means subscription on any change and a database modification is reported.

Subscription on any changes in class

The notification callback is invoked for any changes of class objects including creation of new objects, removing or modification of existing objects.

In C++ to subscribe on any changes in some class the user should to use ::ConfigurationSubscriptionCriteria::add(const std::string&) method. For a class generated by genconffwk the s_class_name attribute can be used, e.g. to subscribe on changes in class dal::Application:

::ConfigurationSubscriptionCriteria c;
c.add(dal::Application::s_class_name);

In Java method conffwk.Subscription.add(String class_name) should be used, e.g. to subscribe on changes in class Application it is necessary to write the following code:

conffwk.Subscription s = new conffwk.Subscription(new TestCallback(db), null);
s.add("Application");

Subscription on object changes

When subscription on object changes has done, the notification callback is invoked for any changes of the objects or it's removing.

In C++ to subscribe on object changes notification the user should to use ::ConfigurationSubscriptionCriteria::add(const ::DalObject&), e.g. to subscribe on changes of an object of the Application class:

::ConfigurationSubscriptionCriteria c;
const dal::Application * app_obj;
c->add(*app_obj);

In Java conffwk.add(DalObject obj) method to be used, e.g.:

dal.Application app = ...;      // some code to get application object
conffwk.Subscription s = new conffwk.Subscription(new TestCallback(db), null);
s.add(app);

7.3. Subscription

To make the actual subscription it is necessary to have a notification callback been implemented and a subscription criteria object. The the method subscribe() to be invoked on the configuration object. For C++ an example is shown below:

  // user-defined callback
void cb(const std::vector<ConfigurationChange *> & changes, void * p) { ... }
 
try {
    // configuration object
  ::Configuration db(...);
 
    // subscription criteria object
  ::ConfigurationSubscriptionCriteria c;
  c.add(dal::Application::s_class_name);
 
    // subscription; if database is changed, the cb can be invoked after this line
  ::CallbackId id = db.subscribe(c, cb);
}
catch(daq::conffwk::Exception & ex) {
  std::cerr << "ERROR: " << ex << std::endl; return 1;
}

For Java above example looks like:

  // user-defined callback
class MyCallback implements conffwk.Callback { ... }
 
  // configuration object
conffwk.Configuration db = new conffwk.Configuration(...);
 
  // subscription criteria object
conffwk.Subscription c = new conffwk.Subscription(new MyCallback(db), null);
c.add("Application");
 
  // subscription; MyCallback::process_changes() can be invoked after this line
db.subscribe(c);

The method unsubscribe() can be used to remove subscription set above. In case of C++ it's parameter is a return value of the subscribe() method (i.e. ::CallbackId value). For Java it's parameter is the subscription object used as parameter of subscribe() method.

8. Algorithms

By default, the generated classes have one-to-one mapping to database schema and DAL objects directly correspond to the database objects. If user wants to add more algorithms on top of the generated DAL without modification of DAL code by hand, he has possibility to define algorithms on top of the OKS class methods.

When a class method is created, user can add it's implementation for different programming languages. To be taken into account by genconffwk, user has to provide C++ and/or Java implementation. Then he has two possibilities:

• declare method prototype, write method implementation in the separate file and add such file when build DAL;
• declare method prototype and write it's implementation in OKS.

The first way is more flexible, but requires more steps when build library. The second way does not require any additional steps when build library, but will require schema modifications to any method's implementation modification.

9. Advanced Code Generation

The OKS methods allow to insert user specific code into generated C++ and Java files and classes. In this case one has to put into method's implementation the code inside special keywords described below:

• BEGIN_HEADER_PROLOGUE and END_HEADER_PROLOGUE (C++ only) Put code into C++ header file before declaration of class containing given method.
• BEGIN_HEADER_EPILOGUE and END_HEADER_EPILOGUE (C++ only) Put code into C++ header file after declaration of class containing given method.
• BEGIN_PUBLIC_SECTION and END_PUBLIC_SECTION Put code into public section of C++ or Java class containing given method.
• BEGIN_PRIVATE_SECTION and END_PRIVATE_SECTION Put code into private section of C++ or Java class containing given method.
• BEGIN_MEMBER_INITIALIZER_LIST and END_MEMBER_INITIALIZER_LIST Put code into initializer list of C++ or Java class containing given method.

The online DAL defines several algorithms (e.g. to find partition, get all applications, to calculate application environment, etc.) and uses first way to implement algorithms. More information can be found in the dal package.