Initial status #3

From our first reflection, we know we need to split the logic of Semantic and Expression. But, from the second reflection, we know we need simple units for computability ease, and a system to handle events on the buffers in and out of those.

This made me think of an old, but well-defined, computation system a school teacher lend me courses about; the sequential systems.

It’s a system defined with discrete alphabets and such as (x,y)->Y where x is the input, y is the current state and Y is the next state.

Put in simple words, it is a finite state machine that transits knowing its current state and its input; implementations work at clock time, have locked buffers and produce deterministic output. Those systems are common for automaton; like coffee machine, red light orchestration or robotic hands in factories.

But, in the meantime, our general diagram shows another part of the system: an output Z dependant of the buffered values (x,y)->Z
We have here interdependencies of both an external output flow and an internal output state in the parameters, but no deadlock relative to this approach; the machines iterates its states and it affects its fluxes, as well as the impact of the input.

From a reactive approach, we can consider the system made of 3 fluxes;

  • one input flux from which x can be considered a subset matching the unit needs
  • one state flux defining internal state of the unit and transiting from y to Y
  • one output flux defined as the response from the unit logic as a new flux Z

There, having both an output flux, result of an undefined process of the input and the internal state, as well as an internal flux; we have an influence but a clear split between the Experience and the Semantic in our data flow. We also know each unit creates 2 new fluxes, as expected, one for the dataflow and the other for the interpretation of the data.

We can also represent that more like 2 processes running in the same unit to define its different flows, like here below with added logic blocks on top of the previous sequential diagram. If we extrapolate its structure, to better serve user experience, we define the unit on the right; receiving both raw data encoded in a given comprenssion format, and graph-oriented data queries.

We can define the input processing, though being in a black box, as a mapping occuring under the command of the internal state and linking the input to an intermediary space to apply some thought-vector like logic for providing us with the output flux.
While the state processing defines the logical rules of the process and memorized them through states. Those can see applied to them some level of abstracted logic to define their transitions. As per grouping, generalized concept with sets of transition can be created (such as counting, driving or writing).

Finally, at higher level, the unit, and its state, is nothing more than an attribute, and its value, of some object which represents a concept in data structures.



Now that we isolated a way to represent Experience/Semantic split in data modeling, we freely expand the unit to serve some more specialized purpose. Here the fields are expected to help handle data and create a process layer, from data POV to user POV.

To wrap my head around the unit structure, I wanted to display it with a GUI. What should I graphically display for each unit? Hmm… let’s be a bit creative in the conception. I can think of a Data Vault model-like for the “higher-than-semantic” level of abstraction. This way, we define the attributes, our units, as satellites around objects, which are empty but linkable concepts, that can themselve operate as per close-to-UML rules between each others.
From each attribute, there is an instantiated State Machine Diagram, similar per type of attribute, corresponding to the internal state logic of the sequential system. The processing corresponding to a functional mapping screen. It ends by the data structure and its conversion system.



Could you imagine those bricks of logic to build anything?
As it is a data processing system, relying on its different logics but mostly on externalized processes, but also on asynchronous reactive approach, it’s probably best to consider it a machine-level service layer. Though I still have hard time thinking of use cases to make the most out of it.

I’d like to start an open source implementation for it, as I believe a modular service layer with those features, can also help fragment backend layer into scripts and more complexes processes running along according to service layer needs. I’m pretty sure it could bring some more insightful experience of any machine, so I’ll try it in Java. And, from RAES, I’ll call it Serf (which is not an acronym this time); it refers to medieval servants, as “Jenkins” is already taken and “Skynet” is way too hasty.


Initial status #2

Last time, we ended up blocked by the nice concept of splitting Semantic and Experience from the data in self-consistent micro-units, but got blocked by the growing complexity of those units. Stuck on this problem, I had to approach it another way around.

For the AGI guys and else: Let’s take driving for instance; we naturally all act as regulators in a closed loop system. The bands are divided in a certain way and we’re expected to stay in between; the visual inputs confirm the driving outputs. For acting this way; we need to specialize part of our bodies to a scope of given functionalities, or behaviors, but we need to do the same for the visual inputs, as we keep vigilant watching the road, which we usually call Focus.

To accomplish this, there are data flows and data structure modeled as a closed loop system. Therefore I introduced a distinction between data logic and processing logic; only the first one is in scope of this approach, the other one is externalized to scripts, services or system calls.

Considering the way to model a closed loop regulated system with simple units; I considered them modular and bidirectional. How could they be used to represent those data? Well there are some MetaData (like encoding or a bit of semantic) and the corresponding Content (bits making sense with a decoding pattern) that needs to be moved from a low level of understanding (unit is mostly content) to a high level of understanding (unit is mostly metadata), like a digestion process of information.

This way, we expect to achieve a continuous transformation from experience (almost only content) to semantic (almost only metadata). Instead of focusing on a dual classification, we want simple units that achieve the state shifting by building the dataflow transformation, and getting as many end points as we have of units to formalize this idea in a simple and modular architecture.

In an autoencoder way of thinking, to keep bidirectionality and consistent conversion, we will have to think of the error as the difference between what gets in and what gets out of our data buffer unit, compared to the output unit. An externalized logic ensure the conversion, but the data consistency requires other blocks such as one for the Error calculation and one for acting as the Controller of the system. As it’s a process which requires data orchestration (basically clock binding), a controller for both units is also required and they need to exchange information with the process controller.

Did we just shifted the complexity? At this point, the system is just made of units, abstraction of externalized process calls, and data logic attributed to units. We have an orchestrator based on process representation and data fields encapsulated; but we can apply a logic to those fields, as to complexify the behavior of our units.

For instance, we can define the DataBuffer with a MetaData;XML field and a Content;SystemFormat field and expect the DataOutput unit to be defined by MetaData;FirstOrderLogic, BusinessRules;ImperativeProgram, Content;CustomFormat for fields. We orchestrate, as we know what we are calling, and we have a contract on what we pass and what we expect.
Also, supposing we have backpressure, the controllers can be supposed independent.

 What do we have here? An Unit that encapsulates different DataLogics which are orchestrated and encoded to an externalized process and, finally, decoded and dispatched to other units.

We’ve got, in short, an unit which produces, at event time, an instance of the dataflows it receives but constrained by data logics  attributes, which are self-containing their own logic rules, and a map connects those fields output flows with a transition logic (weight, threshold, event,…) to something else. This something else being either another unit attribute or a process logic which uses externalized, or system defined, conversion processed from a unit A to a unit B enforced by the contract.

As I was wondering how far this could be abstracted, while still being implementable, many ideas emerged. Such as how far enforcing of fields could be done? Could we use AI to go as far as matching face structure? How could the logic rules be implemented efficiently knowing everything that could be expressed on different abstraction of mathematical logic languages? Should I go for some metagrammar like Backus-Naur Form to define data logics? Or some state machine process?



Well this was still confusing to me, so I used those building blocks to try something else. I wanted to see if the weights could be used to model a general approach of bayesian/frequentist treatment of a situation, but using relative clock times of the units to simulate the weights.

We have all those units, that will be producing an image at their own clock time period, or event-based, and we need to find which value is the most probable one.
For instance; defining the name of a color. Each unit is the output of a given value (color) and a central operator, winner-takes-all, ensure only one value is allowed to pass. If the threshold is count sensitive, we can think of it that way; during 2 clock period where the buffer of the target unit operates, which should then be much larger than the cluster of other units, we have different images that will be generated. The image type having the bigger count at the end of the clock period is, with the WTA mechanism, the one who will be transfered to the input buffer of the target unit.

The weights will be handled by an externalized logic treatment to increase or decrase the clock period relatively to the importance of this unit. Then we have the common part of bayesian and frequentist approach; the threshold mechanism after a logic has weighted the values.

Though we see the limit of the model toolbox here; the result is not fully as per expectation, we’ve lost the pattern from the beginning by refusing complexity, and we have weird behaviors such as weighted units by an externalized process running with the clock period as output.

That gives us a new kind of toolbox where units have a varying clock for both the buffer in and buffer out, that would allow to modulate a gain in the image.

Though we also have those new object called “Connector” which are abstracting the business logic but have both an input and output. While our units have now an input and/or an output (according to clock references) and abstract the data logic.

Defining a rich enough toolbox while accounting for single unit of abstraction seems to really lead to not much from our initial goals perspective. Maybe already existing patterns could be a better gateway to summarize our approach, our goals and the expected results ?


Initial status #1

It’s been a while since I started searching around for an approach that will split the formal and informal between a machine;

  • What could be caught by Semantic, like what we agree to define strictly between ourselves (made of a pronunciation, a written form, a lexical field and a certain pattern of interpretation that multiple individuals can agree upon)
  • What could be caught by Experience, things that can’t be expressed, or only partially, with common nouns and quantifiers, usually due to the inexperience of other experiences to agree upon semantic


I started it in a structured layer approach, naming it RAES for RAES Are Evolving System. And started trying to consider Semantic and Experience a part: as levels of abstraction having different purpose but assuming Semantic is build atop of Experience, out of outbound agreements.

For the AGI guys: I express the idea of event memory bounded by the current interpretation capacity, built itself on top of the richness of Experience.
An example; as a kid, I always identified some bed sheets as a face with a huge heart-shaped nose. After finding them back as an old teenager, I still could see the face, until my vision started to adjust on a couple of swans kissing each other  while their necks were making the heart shape.
Details of the abstract representation weren’t yet in my conscious Experience to understand the eyes weren’t those of the pillow but of two swans.
As my interpretation was wrong, I could have beeen considered in a “confused” state of mind.

So, ok, it should be represented by a system on which everything grows on its own but is built on top of another layer. Thinking about the different functionalities a layer could be taking care of, it started to seem more and more confusing as this doesn’t define much.

Having this way the Experience enclosed between input and semantic would be more as a closed loop system representation in order to develop specialized controller that would either match semantic interpretation with input, or the other way around.
A logic based approach realizing a similar controller for semantic between user and experience was foreseen, as a way to match reality with expectation on higher level abstraction.

Having high-level representation (with semantical meaning, like xml encoded), low-level representation (with set ordering, like fuzzy clusters) and the relation between both of them, this could have been seen as a consistent first approach with the split I wanted to achieve.

So I wondered about how “naming” capacity was acquired; as a fundamental way to tell a part self-consistent subset of data. And I started multiple relational approach. From this layered approach, I started with the semantic as the final input interpretation, the place where the name should be stored and matched with multiple experiences; though the interpretation of what a “cat” is, should be unique in the system, without more context given.

From there, I assumed that for an unique semantic interpretation of experience, there should also be an unique experience, based on an unique instant, as each situation is in the context of a past knowledge that keeps renewing itself for an observer.
After giving a superset to each layer, some adjustment iteration of the model was in order.

Maybe it needed to really consider semantic and experience as different “things” living a part but being able to communicate on both direction. Though I would consider they don’t work on the same kind of data as to really be disconnected units that are matched according to the instant.

For the AGI guys: the representation from the sensors and the representation from the mind are reasonable mismatch. Like assuming the cat from a picture of his tail, observing more violent behavior of an individual due to past experience, building the image of the invisible man from what moves around in the movies, knowing under which pot is the ball,…

Therefore I considered the metadata, and the data, as, respectively, the input object of the semantic layer, and the experience layer. Also, Semantic layer would contain Meaning objects, that would encod and represent the MetaData, as well as the Representation objects, doing the same in Experience layer for Data objects.

But considering that Data is a self-consistent object wasn’t satisfying; as data content have both semantical and experienceful levels of reading. I had to realign on the data as the center of both layers.

This moment, I started to shift a bit the representation. Semantic and Experience were global layers, made of sublayers, made of objects following a given logic. Not sure at this point what could be a concrete use case for this, but my focus was still on a way to properly split the yellow from the white in my egg.
The logic is more of a core approach; data is the most important element of the unit stack; it is generated by crossing meaning and representation, but those should be encapsulated with another data layer expressing the data in their own encoding language. Called “MetaDescription” for Meaning and “MetaData” for Representation. Both directions became from the inside to the outside, and reciprocally.


But now that I was reaching something consistent for the stack, the question got raised; how the hell to use that stack?!

Multiple approaches were though to interact with it, to the point it became confusing.

At this point, I was considering the unit stack like one of those cortical columns that I still have a rest of interest in after reading On Intelligence and Numenta white paper. So I thought of it as a hierarchized bidirectional data flow, with the top loop being another layer, called Programming, which will feedback things like logic rules, loss function, inference conclusion,… based on what the premisces of the unit stack are.

The Programming layer is defined as “How the system should behave knowing this”, the Semantic layer as “What the system interprets it as”, the Experience layer as “What the system compares it to” and the last sensor/motor IO layer as “What the system really sees”.

That was satisfying up to the moment I came back to my previous expectations and figured out that, well, there should be plenty of developers from all the kind on this: UX, Data, Programmer, Logician,… as I also put on the diagram that the Semantic layer could be access from a graphic query language or the Programming layer should be externalized code logic.

It started to be an overbloated stack, basically the unit was now a large complex stack approach. I got confused on the way to rationalize this and moved back a bit to a more modular approach, trying to break that up, and to come with small units that could be based on a P2P network and have the information cloud servers-like encoded through all those small units, I comparabed to threads running in the background.

But, even if I try to split this into more generalized small units, that interact with each other, how would I replicate the functionalities of my higher-order pattern of the relation between Experience and Semantic ?