Actuation, Federation and Interoperability of Context Brokers

Actuation, Federation and
Interoperability of Context Brokers
Advanced and Novel Features available in NGSI-LD 1.6.1
Jason Fox, Senior Technical Evangelist, FIWARE Foundation.

Learning Goals
▪ Overview advanced deployments
▪ Multi-Attributes with datasetId (alternative sources, 1-n relationships)
▪ Scopes (filter based on hierarchies)
▪ Distributed deployments based on registrations (IoT Agents, Context Brokers)
▪ Federated deployments (transparently give applications access to context
information from different players running own Context Brokers)
▪ Tenants (use same Context Broker instance, but isolate databases)
▪ Outlook: New distributed deployment features in NGSI-LD v1.6.1
1

Multi-attributes: Use of datasetId
2

Multi-Attributes with datasetId (1)
Properties
▪ There may be more than one source for the same information, e.g.
speed of the same car from speedometer, navigation system, external sensor
▪ They all provide the speed of the car but they may be systematically different, e.g. the
speedometer shall never show a speed lower than the actual speed in practice it will often
show a slightly higher speed
▪ If a single property value in a Context Broker was arbitrarily updated by the different sensors,
such values would be quite volatile, especially when looking at a temporal evolution, even if
the actual value was relatively stable
It is necessary to distinguish between the values from the different sources
3
“speed": {
"type" : "Property",
"value" : 100
}
“speed": {
"value" : 95
}
“speed": {
"value" : 93
}

Properties
▪ The datasetId has the purpose of distinguishing values from different sources
▪ Property values with different datasetIds are considered distinct values and stored
separately
▪ Property values with a datasetId can thus be updated independently without affecting
values of the same property with a different datasetId
▪ Property values without a datasetId are considered to be the default value
▪ An update without a datasetId updates the default value
▪ A retrieve / query / notification returns all existing values for the same property – as JSON
object if there is a single value, within a container, if there are multiple
4
"speed": {
"value" : 100,
"datasetId": "urn:speedometer"
}
"speed": {
"value" : 95,
"datasetId": "urn:gps"
}
"speed": {
"value" : 93,
"datasetId": "urn:speedcamera"
}

5
▪ Create Entity … Add further speeds
curl -iX POST
'.../ngsi-ld/v1/entities'
--header 'Content-Type: application/json'
--data-raw '{
"id": "urn:HDXX234",
"type": "Vehicle",
"speed": {
"type": "Property",
"value": 100,
"unitCode": "KMH",
}
}'
curl -iX POST
'.../ngsi-ld/v1/entities/urn:HDXX234/attrs'
--data-raw '{
"speed": [
{
"type": "Property",
"value" : 95,
"unitCode": "KMH",
"datasetId": "urn:gps"
},
{
"type": "Property",
"value": 92,
"unitCode": "KMH",
"datasetId": "urn:speedcamera"
}
]
}'

6
▪ Update Speedometer Speed Result
▪ Query Vehicle urn:HDXX234
curl --iX PATCH
'.../ngsi-ld/v1/entities/urn:HDXX234/attrs'
--header 'Content-Type: application/JSON'
--data-raw '{
"type": "Vehicle",
"speed": {
"type": "Property",
"value": 103,
"unitCode": "KMH"
}
}'
{
"type": "Vehicle",
"speed": [
{
"type": "Property",
"datasetId": "urn:gps",
"value": 95,
"unitCode": "KMH"
},
{
"type": "Property",
"datasetId": "urn:speedcamera",
"value": 92,
"unitCode": "KMH"
},
{
"type": "Property",
"datasetId": "urn:speedometer",
"value": 103,
"unitCode": "KMH"
}
]
}
curl -iX GET
'.../ngsi-ld/v1/entities/urn:HDXX234'
--header 'Accept: application/json'

Relationships
▪ In addition to 1:1 relationships, there are also 1:n relationships
▪ Each element of a 1:n relationship can be represented with a datasetId
▪ Each relationship may have different meta information (sub-attributes)
▪ If David becomes the new best friend, the information can be updated
accordingly, using the datasetIds.
7
"friend": {
"type" : "Relationship",
"object" : "John“,
"datasetId": "urn:001"
}
"friend": {
"object" : "Paul",
"datasetId": "urn:002",
}
"friend": {
"object" : "David",
"datasetId": "urn:003",
"bestFriend": {
"value" : true
}
}

Logical Data Separation of Entities: use of scopeQ
8

NGSI-LD Scopes (1)
• Hierarchical structures play an important role for
structuring and organizing our world, e.g.
hierarchical location structures (example on the
left) or organizational structures (e.g. of a
company)
• The special scope property allows giving an entity
a hierarchical scope, e.g. ParqueSur in the
example on the left
• In queries and subscriptions, scopes can be used
for filtering with the scopeQ parameter, e.g.
scopeQ="/Madrid/Gardens/ParqueSur",
which would only match entities that have this
particular scope.
Note: NGSI-LD Scopes are the successor of NGSIv2 Fiware-ServicePath, but
there are important differences
9

NGSI-LD Scopes (2)
• Entities can have multiple hierarchical scopes attached, e.g.
• Scopes are optional and independent of the Entity ID
• Scopes can be used to scope queries and subscriptions
• The scope query language (scopeQ) allows wildcards
• + for an arbitrary value of one hierarchy level, e.g. scopeQ="/CompanyA/+/HR", for matching the
HR groups of all departments of CompanyA
• # matches the given scope and the whole hierarchy of scopes below, e.g.
scopeQ="/Madrid/Gardens/# matches all entities with scope /Madrid/Gardens or any of its
direct or indirect sub-scopes
• /# matches all entities that have any explicit scope attached
• The scope query language (scopeQ) also has logical and and or operators, e.g.
scopeQ="/Madrid/District/Centro,/Madrid/District/Cortes" for all entities whose scope is
/Madrid/District/Centro or /Madrid/District/Cortes
• Scopes of entities can be updated
• Scopes can be registered as part of registrations
"scope":[
"/Madrid/District/Latina",
"/CompanyA/DepartmentX/UnitC"
]
10

NGSI-LD Scopes (3)
11
curl -iX POST
--data-raw '{
"id": "urn:HDMY185",
"type": "Vehicle",
"scope":[
"/Germany/Heidelberg",
],
"speed": {
"type": "Property",
"value": 32,
}
}'
curl -iX POST
--data-raw '{
"id": "urn:BFF2016",
"type": "Vehicle",
"scope":[
"/Germany/Berlin/Charlottenburg",
"/CompanyA/DepartmentX"
],
"speed": {
"type": "Property",
"value": 32,
}
}'

NGSI-LD Scopes (4)
12
curl -iX GET
'.../ngsi-ld/v1/entities?type=Vehicle&scopeQ=/CompanyA'
[]
[
{
"id": "urn:BFF2016",
"type": "Vehicle",
"scope": [
"/Germany/Berlin/Charlottenburg",
"/CompanyA/DepartmentX"
],
…
},
{
"id": "urn:HDMY185",
"type": "Vehicle",
"scope": [
"/Germany/Heidelberg",
],
…
}
]
curl -iX GET
'.../ngsi-ld/v1/entities?type=Vehicle&scopeQ=/CompanyA/%23'
URL encoded Hash

Multi-tenancy: use of the NGSILD-Tenant header
13

Tenants (1)
▪ If the same context broker instance is to be used for multiple different
user (groups), their context information has to be isolated from each
other
▪ The concept for this is called multi-tenancy and each user (group) is
associated with one tenant
▪ NGSI-LD implementations can optionally support multitenancy. In all
requests a tenant can optionally be specified.
▪ In the HTTP binding this is done through the HTTP header “NGSILD-
Tenant”
▪ NGSILD-Tenant is the successor of Fiware-Service in NGSIv2
14

Tenants (2)
▪ The support for tenants can be implicit, i.e. a tenant is created when first used in the
context of a create (entity, subscription, registration) operation. There are currently
no API operations for explicitly creating or deleting tenants.
▪ Principle: if no tenant is specified, there is always a “default tenant” (which does not
have to be created and does not necessarily have to have any information to
provide).
▪ How tenants are implemented, i.e. how isolation is achieved, is up to the
implementation.
▪ Registrations target a tenant (if not, the “default tenant” is assumed) – if the same
context source / broker is to be registered for multiple tenants, multiple registrations
are needed.
15

Accessing Different Tenants
16
curl -iX POST
--data-raw '{
"id": "urn:SZZ546",
"type": "Vehicle",
"speed": {
"type": "Property",
"value": 32,
}
}'
Default Tenant
curl -iX GET
'.../ngsi-ld/v1/entities?type=Vehicle'
[{
"id": "urn:SZZ546",
"type": "Vehicle",
"speed": {
"type": "Property",
"value": 32
}
}]
A named Tenant called "FIWARE"
curl -iX POST
--header 'NGSILD-Tenant: FIWARE'
--data-raw '{
"id": "urn:MYY456",
"type": "Vehicle",
"speed": {
"type": "Property",
"value": 66
}
}'
curl -iX GET
--header 'NGSILD-Tenant: FIWARE'
[{
"id": "urn:MYY456",
"type": "Vehicle",
"speed": {
"type": "Property",
"value": 66
}
}]

Federations, Actuations and Advanced Deployments
Including new Registration features in NGSI-LD 1.6.1
https://www.etsi.org/deliver/etsi_gs/CIM/001_099/009/01.06.01_60/gs_CIM009v010601p.pdf
17

Overview: Advanced Deployments
▪ Simple deployments typically:
• are controlled by a single operator that can
guarantee consistency
• handle a few hundred entities
• are used by a single application or a small set
of applications
… and can be handled by a single Broker /
single database
18
NGSI-LD has features to support advanced deployments
▪ Advanced deployments may:
• handle thousands or even millions of entities
(scalability)
• are controlled by multiple independent operators
• have heterogeneous sources of overlapping
information
• require that only some of the information is shared
• are used by many different applications
… and require interaction between many Brokers
and/or aggregation of information from multiple data
sources

Distributed deployments
NGSI-LD supports distributed deployments with multiple context sources,
which can also be full context brokers
Context brokers, but also IoT agents or other context sources (partially)
implementing the NGSI-LD API, can register with the context registry what
information they can provide
The context broker uses the information from the context registry to access
and aggregate information to be returned to the requesting
As the next level of context brokers can again have context brokers
registered, whole hierarchies of context brokers can be created, possibly
reflecting company structures or geographical structures
19

Federated deployments
▪ Federated deployments are distributed deployments that transparently give
applications access to context information from different players running own
context brokers
• a federated scenario is not technically different from other distributed
scenarios, but the assumption is that it goes across administrative
boundaries, there is no central control that can be assumed
▪ In federated scenarios, the focus typically is on accessing (and possibly
aggregating) context information from multiple context brokers, while the
management of the information (create, update, delete) is done locally in
each domain.
20

NGSI-LD Architectural Roles
Context
Source
Context
Consumer
Query
Subscribe/
Notify
Query
Subscribe/
Notify
Discover
Subscribe/
Notify
Discovery
Register
Discover
Subscribe/
Notify Discovery
Context
Registry
Broker
Context
Producer
Create
Update
Delete
21

Federation and Single Request for different Entities (1)
22
Federation
Broker
Broker 1 Broker 2
Application
Context
Registry
"urn:SZZ546"
"speed"
"urn:BVV987"
"speed"
{
"type": "Vehicle",
"speed": {
"type": "Property",
"value": 100
}
}
{
"id": "urn:SZZ546",
"type": "Vehicle",
"speed": {
"type": "Property",
"value": 32,
"datasetId":"urn:speedometer"
}
}
{
"id": "urn:BVV987",
"type": "Vehicle",
"speed": {
"type": "Property",
"value": 75
}
}
"urn:HDXX234"
"speed"

23
Federation
Broker
Broker 1 Broker 2
Application
Context
Registry
register
register
"urn:vehicle1"
"urn:vehicle2"
"urn:SZZ546"
"speed"
"urn:HDXX234"
"speed"
"urn:BVV987"
"speed"
curl -iX POST
'.../ngsi-ld/v1/csourceRegistrations'
--data-raw '{
"id": "urn:ngsi-ld:ContextSourceRegistration:vehicle1",
"type": "ContextSourceRegistration",
"information": [{
"entities": [{
"type": "Vehicle"
}]
}
],
"endpoint": "http://broker1:9090"
}'
curl iX POST
'.../ngsi-ld/v1/csourceRegistrations'
--data-raw '{
"id": "urn:ngsi-ld:ContextSourceRegistration:vehicle2",
"type": "ContextSourceRegistration",
"information": [{
"entities": [{
"type": "Vehicle"
}]
}
],
"endpoint": "http://broker2:9090"
}'

24
Federation
Broker
Broker 1 Broker 2
Application
Context
Registry
query
query
query
"urn:SZZ546"
"speed"
"urn:HDXX234"
"speed"
"urn:BVV987"
"speed"
"urn:vehicle1"
"urn:vehicle2"
[
{
"id": "urn:BVV987",
"type": "Vehicle",
"speed": {
"type": "Property",
"value": 75
}
},
{
"id": "urn:SZZ546",
"type": "Vehicle",
"speed": {
"type": "Property",
"value": 32
}
},
{
"type": "Vehicle",
"speed": {
"type": "Property",
"value": 100
}
}
]
curl -iX GET

Federation and Single Request for the same Entity (1)
25
Federation
Broker
Broker 1 Broker 2
Application
Context
Registry
"ncc:1701-D"
"class"
"ncc:1701-D"
"bearing"
{
"id": "ncc:1701-D",
"type": "Vehicle",
"speed": {
"type": "Property",
"value": 9.9,
"unit": "WARP"
}
}
{
"id": "ncc:1701-D",
"type": "Vehicle",
"class": {
"type": "Property",
"value": "Galaxy class"
}
}
{
"id": "ncc:1701-D",
"type": "Vehicle",
"bearing": {
"type": "Property",
"value": "14 mark 68"
}
}
"ncc:1701-D"
"speed"
"urn:vehicle1"
"urn:vehicle2"

26
Federation
Broker
Broker 1 Broker 2
Application
Context
Registry
query
query
query
"urn:vehicle1"
"urn:vehicle2"
[
{
"id": "ncc:1701-D",
"type": "Vehicle",
"class": {
"type": "Property",
"value": "Galaxy class"
},
"bearing": {
"type": "Property",
"value": "14 mark 68"
},
"speed": {
"type": "Property",
"value": 9.9,
"unitCode": "WARP"
}
}
]
curl -iX GET
"ncc:1701-D"
"speed"
"ncc:1701-D"
"class"
"ncc:1701-D"
"bearing"

27
Federation
Broker
Broker 1 Broker 2
Application
Context
Registry
"urn:vehicle1"
"urn:vehicle2"
"ncc:1701-D"
"speed"
"ncc:1701-D"
"class"
"ncc:1701-D"
"bearing"
curl --location --
request POST 'http://localhost:9090/ngsi-
ld/v1/entities/ncc:1701-D/attrs'
--data-raw '{
"id": "ncc:1701-D",
"type": "Vehicle",
"speed": [{
"type": "Property",
"value": 21.468,
"unit": "c",
"datasetId": "urn:inspeedoflight"
},
{
"type": "Property",
"value": 6500000000,
"unitCode": "KMH",
"datasetId": "urn:inkmh"
}]
}'
"ncc:1701-D"
"class", "speed"
Add more values to speed property in
Federation Broker, using datasetId

28
Federation
Broker
Broker 1 Broker 2
Application
Context
Registry
query
query
query
"urn:vehicle1"
"urn:vehicle2"
[
{
"id": "ncc:1701-D",
"type": "Vehicle",
…
"speed": [
{
"type": "Property",
"value": 9.9,
"unitCode": "WARP"
},
{
"type": "Property",
"datasetId": "urn:inspeedoflight",
"value": 21.468,
"unit": "c"
},
{
"type": "Property",
"datasetId": "urn:inkmh",
"value": 6500000000,
"unitCode": “KMH"
}
]
}
]
curl --location --request
GET 'http://localhost:9090/ngsi-
ld/v1/entities?type=Vehicle'
"ncc:1701-D"
"speed"
"ncc:1701-D"
"direction"
Broker
1
Federation
Broker
"ncc:1701-D"
"class", "speed"

Tenants in Distributed/Federated Cases:
Tenant Mapping in the Registry
▪ As for other operations, a tenant can be specified in registry operations
▪ Due to isolation between tenants, there are de facto separate registries per tenant (realization
is implementation-dependent)
▪ The available tenants in registered Context Sources or Brokers may be different from those in
the Broker to which the registry belongs therefore a mapping is required
▪ Thus: optional tenant information can be added to registrations (see next slide)
▪ Brokers then use this tenant information from the registrations when interacting with the
registered Context Sources / Brokers (not the tenant information provided in the original
request)
29

Tenant Information in CsourceRegistrations
ADD SECTION NAME
Name Data type Restriction Cardinality Description
id URI At creation time, If it is not provided, it will be
assigned during registration process and
returned to client.
It cannot be later modified in update
operations
0..1 Unique registration identifier. (JSON-LD @id).
There may be multiple registrations per
Context Source, i.e. the id is unique per
registration
type string "ContextSource
Registration"
1 JSON-LD @type
Use reserved type for identifying Context
Source Registration
name string Non-empty string 0..1 A name given to this Context Source
Registration
description string Non-empty string 0..1 A description of this Context Source
Registration
Information RegistrationInfo[] See data type definition in clause 5.2.10. Empty
array (0 length) is not allowed
1 Describes the Entities, Properties and
Relationships for which the Context Source
may be able to provide information
tenant string 0..1 Identifies the tenant that has to be
specified in all requests to the Context
Source that are related to the
information registered in this Context
Source Registration. If not present, the
default tenant is assumed. Should only
be present in systems supporting
multi-tenancy.
[…] […] […] […] […]

Beyond Federations: New distributed deployment features
Focus up to here has been on federated cases of context broker to context
broker operations, which is defined as the default case for NGSI-LD
However, there are also other common distributed deployment scenarios:
▪ Support for actuation (IoT Agents)
▪ Support for “lazy” attributes (IoT Agents)
▪ Explicit distribution of context information, e.g. one context
source for vehicles, another for bicycles
▪ Backup context sources e.g. less frequently updated
▪ Complex Data Sharing scenarios
These require a different subset of operations to the standard
federation case – a full context broker is frequently not required
▪ Security concerns may be explicitly deny certain operations
▪ In other cases, some entities or attributes may only be obtained from
a single specific context source (e.g. device actuation)
▪ Others could be augmented from multiple sources with limited availability
31
NGSI-LD v1.6.1

Distributed Operation Modes (1)
Additive Registrations
A Context Broker is permitted to hold context data about the Entities
and Attributes locally itself, and also obtain data from (possibly multiple)
external sources
▪ An inclusive Context Source Registration specifies that the Context Broker
considers all registered Context Sources as equals and will distribute operations to
those Context Sources even if relevant context data is available directly within the
Context Broker itself (in which case, all results will be integrated in the final
response). This federative and is the default mode of operation.
▪ An auxiliary Context Source Registration never overrides data held directly within
a Context Broker. Auxiliary distributed operations are limited to context information
consumption operations (see clause 5.7). Context data from auxiliary context
sources is only included if it is supplementary to the context data otherwise
available to the Context Broker.
NGSI-LD v1.6.1
additive proxied
inclusive auxiliary exclusive redirect
registration
options
32

Distributed Operation Modes (2)
Proxied Registrations
A Context Broker is not permitted to hold context data about the Entities
and Attributes locally itself. All context data is obtained from the external
registered sources.
▪ An exclusive Context Source Registration specifies that all of the registered
context data is held in a single location external to the Context Broker. The
Context Broker itself holds no data locally about the registered Attributes and no
overlapping proxied Context Source Registrations shall be supported for the
same combination of registered Attributes on the Entity.
An exclusive registration must be fully specified. It always relates to specific
Attributes found on a single Entity. It can be used for actuations
▪ A redirect Context Source Registration also specifies that the registered context
data is held in a location external to the Context Broker, but potentially multiple
distinct redirect registrations can apply at the same time.
NGSI-LD v1.6.1 33
additive proxied
inclusive auxiliary exclusive redirect
registration
options

Limiting Distributed Operations: local
▪ In order to avoid cascading distributed operations, the query parameter local can
be used to limit all operations to be executed on locally available information only.
(Clause 6.3.18)
▪ Context Sources can also put localOnly in the management info, provided in the
Context Source Registration to indicate that, for all requests forwarded to it, local
should be set.
Name Data Type Cardinality Remarks
local boolean 0..1 If local=true then no Context Source Registrations shall
be considered as matching to avoid cascading distributed
operations (see clause 4.3.6.4)
NGSI-LD v1.6.1 34

Context Source Registration
NGSI-LD v1.6.1 35

Supported Distributed Operations in Registration
NGSI-LD v1.6.1
Individual Operations Pre-defined Operation Groups
default
36
⬅ Actuators
⬅ "lazy"
Attributes
⬅ Federations
⬅ CRUD Proxies

Optional Registration Management Info
NGSI-LD v1.6.1 37

Summary (1)
▪ All NGSI-LD brokers (e.g Scorpio, Stellio, Orion-LD) are working towards a common
NGSI-LD specification
▪ The NSGI-LD Specification defines many advanced features, that Context Brokers
support are obliged to support to comply.
• These features are essential for certain advanced use cases, but in general usage of advance
features is not mandatory and usually will not be required when running simpler installations.
▪ Use Multi-Attributes with datasetId for alternative sources, 1-n relationships and
sequential command actuations
▪ Use Scopes to logically separate data. scopeQ filters hierarchical queries
▪ Use Tenants and the NGSILD-Tenant header to isolate context data into separate
databases.
▪ Use Advanced Registration features to alter the scope of distributed operations
39

Summary (2)
Remember, specification runs before implementation:
▪ As of September 2022, The latest specification revision was 1.6.1 and can be found
at: https://www.etsi.org/deliver/etsi_gs/CIM/001_099/009/01.06.01_60/gs_CIM009v010601p.pdf.
▪ Basic operations (e.g. from 1.3, 1.4) are commonly available across all brokers
• Federation has already been achieved between different brokers.
▪ Expect rollout of recently defined advanced distributed operation 1.6.1 features to
arrive in brokers in the coming months based on commercial priorities.
40

Summary (3)
▪ 4 distributed operation modes have been defined :
• inclusive = federation (default)
• exclusive = IoT devices
• redirect = distributed databases
• auxiliary = weak secondary sources
▪ The default list of supported distributed operations is explicitly defined as Federation
(i.e. Query + Subscribe operations)
• No changes required for existing federated environments
▪ For advanced operations and the use of limited partial brokers such as IoT Agents,
switch to using registration mode and the new operations list as soon as available
• Limit unnecessary and unsupported forwarded requests whenever possible.
▪ Registration management info (e.g localOnly) can be used to add further restrictions to
avoid unnecessary traffic, but remember that support for these options is always defined
as best effort only
41

Thank you!
http://fiware.org
Follow @FIWARE on Twitter

Actuation, Federation and Interoperability of Context Brokers

Related slideshows

More Related Content

Actuation, Federation and Interoperability of Context Brokers