How Does it work?
AI Framework
When you drill into each of the Frameworks, you will see that there is a specific map that the AI is guided toward and the performance of that AI can be measured against a standard. Once we have the specific domain as defined by conceptual set of ideas or a given market we can then build that map out to determine if the AI can perform within tolerance in that space.
This answers the question:
How to we measure Performance?
Framework Defined
The need for any system to have a point going forward is always the environment in-which they operate. In cruise missile technology, the flying object had to have the ability to compare its understanding with reality within time. For AI systems working in the real world, regardless of their last point of record, they must be evaluated against a map and within time. These are the components of that mapping and guidance for compliance.
-
4 Dimensional Mapping
-
Dimension Questions
-
Validation of Dimension and Questions
-
Expert Mapping and Correlation of Success
4 Dimensional Mapping
To best establish positioning within a conceptual map we will need to map coordinates against 4 different measures, one from each side. This 4 dimensional map provides a directional mapping of responses as they are captured. Only using two is not enough granular information and more than 4 is too complex at this time.
Dimensional Questions
Within these dimensions for a set of questions on a subject, there will be the need for the AI to pick one of 4 questions. Which each question represents one of the 4 dimensions. With the use of statistics, we can ask 50 to 400 questions that we can map across the 4 Dimensions on a grid as noted above.
Dimensional and Question Validation
Once we have the dimensions and we have the questions defined, we can now do some generalized testing to determine what variations of Dimensions and what type of Questions are the most appropriate and provide a level of assurance.
Expert Mapping and Success Correlation
Once we have defined the playing ground (the map) we define the guidance (the questions) and then we can test the questions to ensure we are mapping to a direction, usually the lower right corner (the validation) we then tackle the last level, that is to work with the experts in the field to do blind testing to determine if the mapping in the real world is aligned, if not, we go back to the start with adjusted assumptions and learned results. We expect 3 to 10 cycles will be required to tune a model that can be used within confidence.